The task of Visual Place Recognition (VPR) is to predict the location of a query image from a database of geo-tagged images. Recent studies in VPR have highlighted the significant advantage of employing pre-trained foundation models like DINOv2 for the VPR task. However, these models are often deemed inadequate for VPR without further fine-tuning on VPR-specific data. In this paper, we present an effective approach to harness the potential of a foundation model for VPR. We show that features extracted from self-attention layers can act as a powerful re-ranker for VPR, even in a zero-shot setting. Our method not only outperforms previous zero-shot approaches but also introduces results competitive with several supervised methods. We then show that a single-stage approach utilizing internal ViT layers for pooling can produce global features that achieve state-of-the-art performance, with impressive feature compactness down to 128D. Moreover, integrating our local foundation features for re-ranking further widens this performance gap. Our method also demonstrates exceptional robustness and generalization, setting new state-of-the-art performance, while handling challenging conditions such as occlusion, day-night transitions, and seasonal variations.

Convolutional neural networks (CNNs) have achieved superior accuracy in many visual-related tasks. However, the inference process through a CNN's intermediate layers is opaque, making it difficult to interpret such networks or develop trust in their operation. In this article, we introduce SIGN method for modeling the network's hidden layer activity using probabilistic models. The activity patterns in layers of interest are modeled as Gaussian mixture models, and transition probabilities between clusters in consecutive modeled layers are estimated to identify paths of inference. For fully connected networks, the entire layer activity is clustered, and the resulting model is a hidden Markov model. For convolutional layers, spatial columns of activity are clustered, and a maximum likelihood model is developed for mining an explanatory inference graph. The graph describes the hierarchy of activity clusters most relevant for network prediction. We show that such inference graphs are useful for understanding the general inference process of a class, as well as explaining the (correct or incorrect) decisions the network makes about specific images. In addition, SIGN provide interesting observations regarding hidden layer activity in general, including the concentration of memorization in a single middle layer in fully connected networks, and a highly local nature of column activities in the top CNN layers.

Latent variables (LVs) represent the origin of many scientific, social, and medical phenomena. While models with only observed variables (OVs) have been well studied, learning a latent variable model (LVM) allowing both types of variables is difficult. Therefore, the assumption of no LVs is usually made, but modeling by ignoring LVs leads to learning a partial/wrong and misleading model that misses the true realm. In recent years, progress has been made in learning LVMs from data, but most algorithms have strong assumptions limiting their scope. Moreover, LVs by nature often change temporally, adding to the challenge and complexity of learning, but current LVM learning algorithms do not account for this. We propose learning locally a causal model in each time slot, and then local to global learning over time slices based on probabilistic scoring and temporal reasoning to transfer the local graphs into a latent dynamic Bayesian network with intra- and inter-slice edges showing causal interrelationships among LVs and between LVs and OVs. Examined using data generated synthetically and of ALS and Alzheimer patients, our algorithm demonstrates high accuracy regarding structure learning, classification, and imputation, and less complexity.

Amyotrophic lateral sclerosis (ALS) is a devastating and incurable disease affecting motor neurons, leading to progressive paralysis and death on average within three to five years from onset. The disease is characterized by highly variable patterns and rates of progression, which pose challenges to developing reliable and accurate ALS disease state prediction models to be used on a daily basis in clinics with little data. To meet these challenges, we suggest domain adaptation from a large, but unfortunately biased, clinical trials database to that of a tertiary care ALS clinic. To evaluate the reliability and accuracy of the suggested paradigm, we examine a naïve approach by which training is based only on the clinical trials data compared with a domain adaptation approach of an initial training using this same data followed by fine-tuning training using the clinic data. We also allow summarization of the clinical longitudinal data to evaluate non-temporal models, e.g., random forest (RF), XGBoost (XGB), and multilayer perceptron (MLP), partially exploiting the dynamic information hidden in patient clinical records, in comparison to the long short-term memory (LSTM) recurrent neural network, fully exploiting the temporal information in the data. First, we notice the XGB outperformance in terms of the ALS disease state prediction error to the RF and MLP, but surprisingly also to the LSTM regardless of prediction time (up to 24 months ahead). We contribute the inferiority of the highly parametrized neural network to the impact of the curse of dimensionality. Second, we show that this error does not significantly increase when the model is trained using only the clinical trials data, especially for LSTM in long prediction times. Finally, we demonstrate that fine-tuning of the clinical trials-based pre-trained model using the clinic data improves the LSTM and MLP performance compared to using solely the clinical trials or clinic data.

A major challenge in data stream applications is the change in the target variable over time in unexpected ways, a phenomenon called concept drift (CD). Another challenge is the emergence of novel classes, soliciting novelty detection (ND) by, e.g., one-class or semi-supervised classification. But, in online ND, these two challenges interfere with each other although they should be dealt with jointly. We present the cluster drift detection (CDD) algorithm that, using a single hyper-parameter, performs offline clustering to learn the diverse normal profile, and detects online whether a never-seen-before example is novel or normal using a multivariate statistical test. If it is normal, the CDD uses this example to update the normal-profile cluster, enabling continuous CD monitoring. Experimental results using popular real-world and synthetic data sets, as well as a precision agriculture data set of banana plants under water stress and a COVID-19 data set demonstrate that the CDD algorithm: 1) distinguishes between normal and novel concepts more accurately than state-of-the-art algorithms, 2) provides information about why specific novel concepts are misdetected, and 3) is more robust to the complexity, drift, and noise in the problem than other algorithms.

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease, lasting from the day of onset until death. Factors such as the progression rate and pattern of the disease vary greatly among patients, making it difficult to achieve accurate predictions about ALS. To accurately predict ALS disease state and deterioration, we propose a novel approach that combines: a) sequence clustering based on dynamic time warping for separation among patients with diverse ALS deterioration patterns, b) sequential pattern mining for discovery of deterioration changes that patients of the same type may have in common, and c) deterioration-based patient next-state prediction. Using a clinical dataset, we demonstrate the advantage of the proposed approach in terms of classification accuracy and deterioration detection compared to other classification methods and temporal models such as long short-term memory.

Although recent studies have shown that a Bayesian network classifier (BNC) that maximizes the classification accuracy (i.e., minimizes the 0/1 loss function) is a powerful tool in knowledge representation and classification, this classifier focuses on the majority class, is usually uninformative about the distribution of misclassifications, and is insensitive to error severity (making no distinction between misclassification types). We propose to learn a BNC using an information measure (IM) that jointly maximizes classification and information, and evaluate this measure using various databases. We show that an IM-based BNC is superior to BNCs learned using other measures, especially for ordinal classification and imbalanced problems, and does not fall behind state-of-the-art algorithms with respect to accuracy and amount of information provided.

We develop methodologies and apply machine-learning algorithms to a database of ALS patients to expose and model underlying mechanisms and relations in the disease. We view the disease state as an ordinal variable (with values between 4 for normal function and 0 for complete loss of function), and show that ordinal classification applied to the data has an advantage over classification that does not utilize the ordinal nature of the domain. To identify important physiological and lab test variables in relation to patient functionality, we rank variables with a decision tree that predicts future disease state using current and past variable instantiations. In addition, we cluster data of patient functionalities in performing daily tasks into higher level groupings of body segments and show how certain variables relate more concretely to certain groupings than to others, thus reducing the dimensionality of the disease state representation in a natural manner that was found to be medically interpretable. Finally, we learn Bayesian networks to detect predictors within the Markov blanket of the disease-state variable and to expose relations among the predictors and with the disease state, as well as to identify value combinations of the predictors that distinguish severe and mild patients.

Identification of latent variables that govern a problem and the relationships among them, given measurements in the observed world, are important for causal discovery. This identification can be accomplished by analyzing the constraints imposed by the latents in the measurements. We introduce the concept of pairwise cluster comparison (PCC) to identify causal relationships from clusters of data points and provide a two- stage algorithm called learning PCC (LPCC) that learns a latent variable model (LVM) using PCC. First, LPCC learns exogenous latents and latent colliders, as well as their observed descendants, by using pairwise comparisons between data clusters in the measurement space that may explain latent causes. Since in this first stage LPCC cannot distinguish endogenous latent non-colliders from their exogenous ancestors, a second stage is needed to extract the former, with their observed children, from the latter. If the true graph has no serial connections, LPCC returns the true graph, and if the true graph has a serial connection, LPCC returns a pattern of the true graph. LPCC's most important advantage is that it is not limited to linear or latent-tree models and makes only mild assumptions about the distribution. The paper is divided in two parts: Part I (this paper) provides the necessary preliminaries, theoretical foundation to PCC, and an overview of LPCC; Part II formally introduces the LPCC algorithm and experimentally evaluates its merit in different synthetic and real domains. The code for the LPCC algorithm and data sets used in the experiments reported in Part II are available online.

It is important for causal discovery to identify any latent variables that govern a problem and the relationships among them, given measurements in the observed world. In Part I of this paper, we were interested in learning a discrete latent variable model (LVM) and introduced the concept of pairwise cluster comparison (PCC) to identify causal relationships from clusters of data points and an overview of a two-stage algorithm for learning PCC (LPCC). First, LPCC learns exogenous latent variables and latent colliders, as well as their observed descendants, by using pairwise comparisons between data clusters in the measurement space that may explain latent causes. Second, LPCC identifies endogenous latent non- colliders with their observed children. In Part I, we showed that if the true graph has no serial connections, then LPCC returns the true graph, and if the true graph has a serial connection, then LPCC returns a pattern of the true graph. In this paper (Part II), we formally introduce the LPCC algorithm that implements the PCC concept. In addition, we thoroughly evaluate LPCC using simulated and real-world data sets in comparison to state-of-the-art algorithms. Besides using three real-world data sets, which have already been tested in learning an LVM, we also evaluate the algorithms using data sets that represent two original problems. The first problem is identifying young drivers' involvement in road accidents, and the second is identifying cellular subpopulations of the immune system from mass cytometry. The results of our evaluation show that LPCC improves in accuracy with the sample size, can learn large LVMs, and is accurate in learning compared to state-of- the-art algorithms. The code for the LPCC algorithm and data sets used in the experiments reported here are available online.

Solving a multiclass classification task using a small imbalanced database of patterns of high dimension is difficult due to the curse-of-dimensionality and the bias of the training toward the majority classes. Such a problem has arisen while diagnosing genetic abnormalities by classifying a small database of fluorescence in situ hybridization signals of types having different frequencies of occurrence. We propose and experimentally study using the cytogenetic domain two solutions to the problem. The first is hierarchical decomposition of the classification task, where each hierarchy level is designed to tackle a simpler problem which is represented by classes that are approximately balanced. The second solution is balancing the data by up-sampling the minority classes accompanied by dimensionality reduction. Implemented by the naive Bayesian classifier or the multilayer perceptron neural network, both solutions have diminished the problem and contributed to accuracy improvement. In addition, the experiments suggest that coping with the smallness of the data is more beneficial than dealing with its imbalance.

Signal segmentation and classification of fluorescence in situ hybridization (FISH) images are essential for the detection of cytogenetic abnormalities. Since current methods are limited to dot-like signal analysis, we propose a methodology for segmentation and classification of dot and non-dot-like signals. First, nuclei are segmented from their background and from each other in order to associate signals with specific isolated nuclei. Second, subsignals composing non-dot-like signals are detected and clustered to signals. Features are measured to the signals and a subset of these features is selected representing the signals to a multiclass classifier. Classification using a naive Bayesian classifier (NBC) or a multilayer perceptron is accomplished. When applied to a FISH image database, dot and non-dot-like signals were segmented almost perfectly and then classified with accuracy of approximately 80% by either of the classifiers.

In this paper, we modify the fuzzy ARTMAP (FA) neural network (NN) using the Bayesian framework in order to improve its classification accuracy while simultaneously reduce its category proliferation. The proposed algorithm, called Bayesian ARTMAP (BA), preserves the FA advantages and also enhances its performance by the following: (1) representing a category using a multidimensional Gaussian distribution, (2) allowing a category to grow or shrink, (3) limiting a category hypervolume, (4) using Bayes' decision theory for learning and inference, and (5) employing the probabilistic association between every category and a class in order to predict the class. In addition, the BA estimates the class posterior probability and thereby enables the introduction of loss and classification according to the minimum expected loss. Based on these characteristics and using synthetic and 20 real-world databases, we show that the BA outperformes the FA, either trained for one epoch or until completion, with respect to classification accuracy, sensitivity to statistical overlapping, learning curves, expected loss, and category proliferation.

We propose and investigate the fuzzy ARTMAP neural network in off and online classification of fluorescence in situ hybridization image signals enabling clinical diagnosis of numerical genetic abnormalities. We evaluate the classification task (detecting a several abnormalities separately or simultaneously), classifier paradigm (monolithic or hierarchical), ordering strategy for the training patterns (averaging or voting), training mode (for one epoch, with validation or until completion) and model sensitivity to parameters. We find the fuzzy ARTMAP accurate in accomplishing both tasks requiring only very few training epochs. Also, selecting a training ordering by voting is more precise than if averaging over orderings. If trained for only one epoch, the fuzzy ARTMAP provides fast, yet stable and accurate learning as well as insensitivity to model complexity. Early stop of training using a validation set reduces the fuzzy ARTMAP complexity as for other machine learning models but cannot improve accuracy beyond that achieved when training is completed. Compared to other machine learning models, the fuzzy ARTMAP does not loose but gain accuracy when overtrained, although increasing its number of categories. Learned incrementally, the fuzzy ARTMAP reaches its ultimate accuracy very fast obtaining most of its data representation capability and accuracy by using only a few examples. Finally, the fuzzy ARTMAP accuracy for this domain is comparable with those of the multilayer perceptron and support vector machine and superior to those of the naive Bayesian and linear classifiers.

Fast and accurate analysis of fluorescence in-situ hybridization images for signal counting will depend mainly upon two components: a classifier to discriminate between artifacts and valid signals of several fluorophores (colors), and well discriminating features to represent the signals. Our previous work (2001) has focused on the first component. To investigate the second component, we evaluate candidate feature sets by illustrating the probability density functions and scatter plots for the features. The analysis provides first insight into dependencies between features, indicates the relative importance of members of a feature set, and helps in identifying sources of potential classification errors. Class separability yielded by different feature subsets is evaluated using the accuracy of several neural network-based classification strategies, some of them hierarchical, as well as using a feature selection technique making use of a scatter criterion. Although applied to cytogenetics, the paper presents a comprehensive, unifying methodology of qualitative and quantitative evaluation of pattern feature representation essential for accurate image classification. This methodology is applicable to many other real-world pattern recognition problems.

Fast and accurate analysis of fluorescence in-situ hybridisation (FISH) images depends upon two components: a classifier to discriminate between artefacts and valid signal data, and well-discriminating features to represent the signals. After processing the image, we evaluate candidate feature sets by illustrating the probability density functions and scatter plots for the features. This analysis indicates the relative importance of members of a feature set, helps in identifying sources of potential classification errors and recommends several size, intensity and hue-based features for representing FISH signals. The recommendation is assessed by the probability of misclassification of a neural network based hierarchical strategy, and also by a feature selection technique making use of a class separability criterion. Represented by these features, nearly 90 % of valid signals and artefacts within a set of 400 test images are correctly classified. 1

The application of neural networks (NNs) to automatic analysis of chromosome images is investigated in this paper. All aspects of the analysis, namely segmentation, feature description, selection and extraction, and classification, are studied. As part of the segmentation process, the separation of clusters of partially occluded chromosomes, which is the critical stage that state-of-the-art chromosome analyzers usually fail to accomplish, is performed. First, a moment representation of the image pixels is clustered to create a binary image without a need for threshold selection. Based on the binary image, lines connecting cut points imply possible separations. These hypotheses are verified by a multilayer perceptron (MLP) NN that classifies the two segments created by each separating line. Use of a classification-driven segmentation process gives very promising results without a need for shape modeling or an excessive use of heuristics. In addition, an NN implementation of Sammon's mapping using principal component based initialization is applied to feature extraction, significantly reducing the dimensionality of the feature space and allowing high classification capability. Finally, by applying MLP based hierarchical classification strategies to a well-explored chromosome database, we achieve a classification performance of 83.6%. This is higher than ever published on this database and an improvement of more than 10% in the error rate. Therefore, basing a chromosome analysis on the NN-based techniques that are developed in this research leads toward a completely automatic human chromosome analysis.

Feature extraction for exploratory data projection aims for data visualization by a projection of a high-dimensional space onto two or three-dimensional space, while feature extraction for classification generally requires more than two or three features. We study extraction of more than three features, using neural network (NN) implementation of Sammon's mapping to be applied for classification. The experiments reveal that Sammon's mapping, the multilayer perceptron (MLP) and the principal component analysis (PCA) based feature extractors yield similar classification performance. We investigate a random- and PCA-based initializations of Sammon's mapping. When the PCA is applied to initialize Sammon's projection, only one experiment is required and only a fraction of the training period is needed to achieve performance comparable with that of the random initialization. Furthermore, the PCA based initialization affords better human chromosome classification performance even when using a few eigenvectors.

A multilayer perceptron (MLP) neural network (NN) has been studied for human chromosome classification. Only 10-20 examples were required for the MLP NN to reach its ultimate performance classifying chromosomes of 5 types. The empirical dependence of the entropic error on the number of examples was found to be highly comparable to the 1/t function. The principal component analysis (PCA) was used, both for network initialization and for feature reduction purposes. The PCA demonstrated the importance of retaining most of the image information whenever small training sets are used. The MLP NN classifier outperformed the Bayes piecewise classifier for all the cases tested. The MLP classifier was found to be almost unsusceptible to the ratio of the number of training vectors to the number of features, whereas the piecewise classifier was highly dependent on this ratio.

A multilayer perceptron (MLP) neural network (NN) was used for human chromosome classification. The significance of relevant chromosome features to the classification procedure was evaluated using a feature selection mechanism. It yielded the benefit of using only a part of the available features to get performance close to the ultimate one, classifying chromosomes of 5 types. Only 10-20 examples were required for the MLP NN classifier to reach its supreme performance disregarding the number of features used. Furthermore, the empirical entropic error of the classifier was found to be highly comparable to the 1/t function that is a universal learning curve.