Mitosis is a crucial process ensuring the faithful transmission of the genetic information stored in the cell nucleus. Aberrations in this intricate process pose a significant threat to an organism’s health, leading to conditions like cancer and various diseases. Hence, the study of mitosis holds paramount importance. Recent investigations have involved manual and semi-automated analyses of time-lapse microscopy images to understand mitosis better. This paper introduces an approach for predicting mitosis stages, employing a Convolutional Neural Network (CNN) as the initial feature extractor, followed by a Graph Neural Network (GNN) for predicting cell cycle states. A distinctive timestamp is incorporated into the feature vectors, treating this information as a graph to leverage internal interactions for predicting the subsequent cell state. To assess performance, experiments were conducted on three datasets, demonstrating that our method exhibits comparable efficacy to state-of-the-art techniques.

We propose a comprehensive end-to-end pipeline designed for the detection, subtyping, and grading of tumors. Our proposed methodology initiates the generation of a heat map, indicating the severity of the tumor. Subsequently, the identification of the most critical patches is conducted based on the probability scores. These identified patches are then directed to a grade prediction network. A distinctive aspect of our research lies in being the first to explore an end-to-end pipeline for both heat map generation and grading prediction. Our experiments were conducted leveraging the public, The Cancer Genome Atlas (TCGA) repository, focusing specifically on renal cancer. We introduced additional patch-level labels to improve the model performance. The generation of tumor heat maps targeted three primary cancer subtypes: clear cell, papillary, and chromo-phobe. To enhance our approach, we implemented center-loss and introduced a method aimed at refining the quality of patches. The experimental outcomes highlight superior performance compared to state-of-the-art method. This research contributes to the advancement of tumor detection and grading, emphasizing the significance of an integrated approach for heat map generation and grading prediction.

Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models.

Cell division, or mitosis, guarantees the accurate inheritance of the genomic information kept in the cell nucleus. Malfunctions in this process cause a threat to the health and life of the organism, including cancer and other manifold diseases. It is therefore crucial to study in detail the cell-cycle in general and mitosis in particular. Consequently, a large number of manual and semi-automated time-lapse microscopy image analyses of mitosis have been carried out in recent years. In this paper, we propose a method for automatic detection of cell-cycle stages using a recurrent neural network (RNN). An end-to-end model with center-cell focus tracker loss, and classification loss is trained. The evaluation was conducted on two time-series datasets, with 6-stages and 3-stages of cell splitting labeled. The frame-to-frame accuracy was calculated and precision, recall, and F1-Score were measured for each cell-cycle stage. We also visualized the learned feature space. Image reconstruction from the center-cell focus module was performed which shows that the network was able to focus on the center-cell and classify it simultaneously. Our experiments validate the superior performance of the proposed network compared to a classifier baseline.

In this paper, we propose an approach for learning binary hash codes for image retrieval. Canonical Correlation Analysis (CCA) is used to design two loss functions for training a neural network such that the correlation between the two views to CCA is maximum. The main motivation for using CCA for feature space learning is that dimensionality reduction is possible and short binary codes could be generated. The first loss maximizes the correlation between the hash centers and the learned hash codes. The second loss maximizes the correlation between the class labels and the classification scores. In this paper, a novel weighted mean and thresholding-based hash center update scheme for adapting the hash centers is proposed. The training loss reaches the theoretical lower bound of the proposed loss functions, showing that the correlation coefficients are maximized during training and substantiating the formation of efficient feature space for retrieval. The measured mean average precision shows that the proposed approach outperforms other state-of-the-art methods.

We propose a method for learning the Linear Discriminant Analysis (LDA) using a Siamese Neural Network (SNN) architecture for learning a low dimensional image descriptor. The novelty of our work is that we learn the LDA projection matrix between the final fully-connected layers of an SNN. An SNN architecture is used since the proposed loss maximizes the Kullback-Leibler divergence between the feature distributions from the two branches of an SNN. The network learns an optimized feature space having inherent properties pertaining to the learning of LDA. The learned image descriptors are a) low-dimensional, b) have small intra-class variance, c) large inter-class variance, and d) can distinguish the classes with linear decision hyperplanes. The proposed method has the advantage that LDA learning happens end-to-end. We measured the classification accuracy in the three datasets MNIST, CIFAR-10, and STL-10 and compared the performance with other state-of-the-art methods. We also measured the KL divergence between the class pairs and visualized the projections of feature vectors along the learned discriminant directions.

In this paper, a novel approach for learning a low-dimensional optimized feature space for image retrieval with minimum intra-class variance and maximum inter-class variance is proposed. The classical approach of Linear Discriminant Analysis (LDA) is generally used for generating an optimized low-dimensional feature space for single-labeled images. Since image retrieval involves images with multiple objects, LDA cannot be directly used for dimensionality reduction and feature space optimization. This problem is addressed by utilizing the relationship between LDA and Canonical Correlation Analysis (CCA) eigenvalues to generate an optimized feature space for both single-labeled and multi-labeled images. A CCA-based network architecture which correlates the low-dimensional feature vectors with the image label vectors is proposed. We design a novel loss function such that the correlation coefficients of CCA are maximized. Our experiments prove that we could train the neural network to reach the theoretical lower bound of loss corresponding to the negative sum of the correlation coefficients. Once the optimized feature space is generated, feature vectors are binarized with the Iterative Quantization (ITQ) approach. Finally, we propose an ensemble network to generate binary codes of desired bit length for retrieval. The measurement of mean average precision shows that the proposed approach outperforms the retrieval results of other single-labeled and multi-labeled image retrieval benchmarks at same bit numbers in a considerable number of cases.

In this work, we target a known problem in representation learning that is: beyond coarse classification, how can we better model fine-grained categorization? To address this problem, we introduce Deep Subclass Linear Discriminant Analysis (DeepSDA), which utilizes intra-class variation and inter-class similarity during training. We could achieve multimodal classification by maximizing the ratio of between-subclass scatter matrix and within-subclass scatter matrix. We maximize the eigenvalues along the discriminative eignevector directions. Hence the deep neural network is able to learn more discriminative representation space and thus has higher class separation in the linearly separable latent space. We show that DeepSDA leads to significant improvements on diverse fine-grained categorization and attribute learning benchmarks.

We propose a novel method for content based image retrieval based on the features extracted from the convolutional layers of the deep neural network architecture. Some of the popular approaches form the feature vectors from the fully connected layers of the convolutional neural networks or directly concatenate the features from the convolutional layers. However, the main problem with the use of feature vectors from fully connected layers is that the spatial information about the objects are lost. This motivated us to use the features from the convolutional layer. We incorporate a pyramid pooling based approach to form more compact and location invariant feature vectors. We have measured the Mean Average Precision (MAP) on benchmark databases such as the Holidays and Oxford5K datasets using features extracted from the AlexNet model. The proposed method gives better retrieval results compared to other state-of-the-art approaches which use feature vectors from fully connected layers and convolutional layers without spatial pooling.

In content-based image retrieval systems, visual content of the image is the criterion for measuring image similarity. We propose a method to solve the problem of loss of spatial information of objects when local descriptors from an image with multiple objects are aggregated to form a global representation. In our approach, after saliency-based spatial partitioning, local feature descriptors from distinct sub-regions are aggregated to form a bag of Fisher Vectors representation. This helps in suppressing the information from background clutter in scenes while forming the global descriptor. The retrieval performance was evaluated in synthetic and real datasets. The evaluation results show that the bag of Fisher Vectors representation gives better performance compared to baseline approach using Fisher Vectors.

With the growth in multimedia data, it is the need of the hour to have methods for efficient storage and quick retrieval. In this work, we propose an approach for learning binary codes for fast image retrieval. We use a siamese architecture with two parallel feed forward branches but with a shared weight for the generation of binary codes. The training data is divided into similar and dissimilar pairs. The network tries to learn the weights such that it minimizes the distance between similar image pairs and maximizes the distance between dissimilar image pairs. The binary codes are formed by squashing the neural network output through a sigmoid activation function. The training with sigmoid hashing constrains the output of each node in the final fully connected layer to either 0 or 1. We have compared the retrieval performance of our approach with other state-of-the-art hashing methods and our method shows significant improvement.

We propose to employ bilateral filters to solve the problem of edge detection. The proposed methodology presents an efficient and noise robust method for detecting edges. Classical bilateral filters smooth images without distorting edges. In this paper, we modify the bilateral filter to perform edge detection, which is the opposite of bilateral smoothing. The Gaussian domain kernel of the bilateral filter is replaced with an edge detection mask, and Gaussian range kernel is replaced with an inverted Gaussian kernel. The modified range kernel serves to emphasize dissimilar regions. The resulting approach effectively adapts the detection mask according as the pixel intensity differences. The results of the proposed algorithm are compared with those of standard edge detection masks. Comparisons of the bilateral edge detector with Canny edge detection algorithm, both after non-maximal suppression, are also provided. The results of our technique are observed to be better and noise-robust than those offered by methods employing masks alone, and are also comparable to the results from Canny edge detector, outperforming it in certain cases.

We propose a method for ridge detection at different widths using second-order Gaussian derivative masks. The width of the ridge extracted varies depending on the mask size and its parameter, σ. In the proposed method, the ridge orientations are estimated as an initial step by finding the zerocrossings of the first derivative of the second directional derivative. In order to compute the orientations from discrete samples of the image, we make use of the recently popularized Savitzky-Golay (S-G) filter. Once the directions are estimated, ridge detection is accomplished by steering a second-order Gaussian kernel, which closely approximates the ideal ridge template, in the computed directions. The method is computationally effective on two accounts: (1) The ridge orientations are determined efficiently using S-G filtering; and (2) Once the orientations are estimated, the steerability property is used to detect ridges. The output of the ridge detector is then improved using non-maximal suppression and hysteresis thresholding. The results obtained are compared with an efficient benchmark method for ridge extraction.