Deep neural networks (DNNs), in addition to great power, exhibit significant weaknesses, with one of the most notable being the susceptibility to adversarial attacks by carefully crafted, minimally perturbed, adversarial examples. Despite numerous efforts towards creating reliable detectors and defense mechanisms, the underlying factors that make such attacks possible have not yet been fully understood. In this paper, we further explore the use of local intrinsic dimensionality (LID), and, for the first time, hubness, as indicators for detection of adversarial examples. Furthermore, we involve neural tangent kernels (NTKs) in this task, showing that NTK-induced distance measures offer more information on how to detect adversarial examples, through both LID and hubness, than distances computed on raw input data or one network layer. Furthermore, we combine LID and hubness as features into a well-performing classifier that exhibits better detection accuracy than any of the two features used on their own. Finally, we combine the LID and hubness features with state-of-the-art adversarial attack detectors, demonstrating their general usefulness for the task. Our findings offer insight into the properties and nature of adversarial examples of various types, paving the way to the construction of better detectors and, possibly, the construction of models that are more robust to such attacks.

Abstract This study presents a novel approach to enhance the accuracy of automatic classification systems for airborne pollen particles by integrating domain adaptation techniques. Our method incorporates expert-verified measurements into the convolutional neural network (CNN) training process to address the discrepancy between laboratory test data and real-world environmental measurements. We systematically fine-tuned CNN models, initially developed on standard reference datasets, with these expert-verified measurements. A comprehensive exploration of hyperparameters was conducted to optimize the CNN models, ensuring their robustness and adaptability across various environmental conditions and pollen types. Empirical results indicate a significant improvement, evidenced by a 22.52% increase in correlation and a 38.05% reduction in standard deviation across 29 cases of different pollen classes over multiple study years. This research highlights the potential of domain adaptation techniques in environmental monitoring, particularly in contexts where the integrity and representativeness of reference datasets are difficult to verify.

We present a nonparametric method for outlier detection that takes full account of local variations in intrinsic dimensionality within the dataset. Using the theory of Local Intrinsic Dimensionality (LID), our ‘dimensionality-aware’ outlier detection method, DAO, is derived as an estimator of an asymptotic local expected density ratio involving the query point and a close neighbor drawn at random. The dimensionality-aware behavior of DAO is due to its use of local estimation of LID values in a theoretically-justified way. Through comprehensive experimentation on more than 800 synthetic and real datasets, we show that DAO significantly outperforms three popular and important benchmark outlier detection methods: Local Outlier Factor (LOF), Simplified LOF, and kNN.

BACKGROUND: Microbiome dysbiosis has recently been associated with different diseases and disorders. In this context, machine learning (ML) approaches can be useful either to identify new patterns or learn predictive models. However, data to be fed to ML methods can be subject to different sampling, sequencing and preprocessing techniques. Each different choice in the pipeline can lead to a different view (i.e., feature set) of the same individuals, that classical (single-view) ML approaches may fail to simultaneously consider. Moreover, some views may be incomplete, i.e., some individuals may be missing in some views, possibly due to the absence of some measurements or to the fact that some features are not available/applicable for all the individuals. Multi-view learning methods can represent a possible solution to consider multiple feature sets for the same individuals, but most existing multi-view learning methods are limited to binary classification tasks or cannot work with incomplete views. RESULTS: We propose irBoost.SH, an extension of the multi-view boosting algorithm rBoost.SH, based on multi-armed bandits. irBoost.SH solves multi-class classification tasks and can analyze incomplete views. At each iteration, it identifies one winning view using adversarial multi-armed bandits and uses its predictions to update a shared instance weight distribution in a learning process based on boosting. In our experiments, performed on 5 multi-view microbiome datasets, the model learned by irBoost.SH always outperforms the best model learned from a single view, its closest competitor rBoost.SH, and the model learned by a multi-view approach based on feature concatenation, reaching an improvement of 11.8% of the F1-score in the prediction of the Autism Spectrum disorder and of 114% in the prediction of the Colorectal Cancer disease. CONCLUSIONS: The proposed method irBoost.SH exhibited outstanding performances in our experiments, also compared to competitor approaches. The obtained results confirm that irBoost.SH can fruitfully be adopted for the analysis of microbiome data, due to its capability to simultaneously exploit multiple feature sets obtained through different sequencing and preprocessing pipelines.

Machine learning systems are vulnerable to adversarial attack. By applying to the input object a small, carefully-designed perturbation, a classifier can be tricked into making an incorrect prediction. This phenomenon has drawn wide interest, with many attempts made to explain it. However, a complete understanding is yet to emerge. In this paper we adopt a slightly different perspective, still relevant to classification. We consider retrieval, where the output is a set of objects most similar to a user-supplied query object, corresponding to the set of k-nearest neighbors. We investigate the effect of adversarial perturbation on the ranking of objects with respect to a query. Through theoretical analysis, supported by experiments, we demonstrate that as the intrinsic dimensionality of the data domain rises, the amount of perturbation required to subvert neighborhood rankings diminishes, and the vulnerability to adversarial attack rises. We examine two modes of perturbation of the query: either `closer' to the target point, or `farther' from it. We also consider two perspectives: `query-centric', examining the effect of perturbation on the query's own neighborhood ranking, and `target-centric', considering the ranking of the query point in the target's neighborhood set. All four cases correspond to practical scenarios involving classification and retrieval.

The problem of time-series classification witnessed the application of many techniques for data mining and machine learning, including neural networks, support vector machines, and Bayesian approaches. Somewhat surprisingly, the simple 1-nearest neighbor (1NN) classifier, in combination with the Dynamic Time Warping (DTW) distance measure, is still competitive and not rarely superior to more advanced classification methods, which includes the majority-voting k-nearest neighbor (kNN) classifier. In this paper we focus on the kNN classifier combined with the inverse-squared weighting scheme, and its interaction with constrained DTW distance. By performing experiments on the entire UCR Time Series Classification Archive we show that with proper selection of the constraint parameter r and neighborhood size k, inverse-square weighted kNN consistently outperforms 1NN.

The complexity of IoT systems and tasks that are put before them require shifts in the way resources and service provisioning are managed. The concept of edge computing is introduced to enhance IoT systems' scalability, reactivity, efficiency, and privacy. In this paper, we present an edge computing solution for resource management of context-aware decision-making processes distributed between IoT gateways. The solution performs decision-making process management for smart actuation, based on analysis of sensory data streams, and context-informed edge computing resource and service provisioning management based on topology and operational changes. Our architectural solution showcases the first version of a Resource Management Framework - a generic framework for software resource orchestration best-suited to IoT platforms with event-driven, publish-subscribe communication mechanisms. Proof of concept experiments that are executed in a simulated edge computing testbed validate our solution's performance in improving the resilience and responsiveness of the edge computing system when there are operational and topology changes. Furthermore, the framework addresses the recovery of failed decision-making processes, impacting the overall health of the underlying IoT system.

In the domain of time-series classification, one simple but persistently successful method is the 1-nearest neighbour (1NN) classifier coupled with an elastic distance measure such as Dynamic Time Warping (DTW). In this paper we evaluate the performance of DTW when constrained using the Itakura parallelogram, and compare it with the more commonly used Sakoe-Chiba band, as well as with the unconstrained DTW. Results show that although the Itakura parallelogram is generally inferior to the Sakoe-Chiba band, it is still superior to unconstrained DTW. Furthermore, on individual data sets the Itakura parallelogram can produce superior results, warranting further investigation into the merits of its use with DTW and other elastic distance measures for time-series classification.

The overarching connectivity of "things" in the Internet of Things presents an appealing environment for innovation and business ventures, but also brings a certain set of security challenges. Engineering secure Internet of Things systems requires addressing the peculiar circumstances under which they operate: constraints due to limited resources, high node churn, decentralized decision making, direct interfacing with end users etc. Thus, techniques and methodologies for building secure and robust Internet of Things systems should support these conditions. In this paper, we are presenting a description of the CAAVI-RICS framework, a novel security review methodology tightly coupled with distributed, Internet of Things and fog computing systems. With CAAVI-RICS we are exploring credibility, authentication, authorization, verification, and integrity (CAAVI) through explaining the rationale, influence, concerns and security solutions (RICS) that accompany them. Our contribution is a thorough systematic categorization and rationalization of security issues, covering the security landscape of Internet of Things/fog computing systems, as well as contributing to the discussion on the aspects of fog computing security and state-of-the-art solutions. Specifically, in this paper we explore the Authentication in Internet of Things systems through the RICS review methodology.

Accurate estimation of Intrinsic Dimensionality (ID) is of crucial importance in many data mining and machine learning tasks, including dimensionality reduction, outlier detection, similarity search and subspace clustering. However, since their convergence generally requires sample sizes (that is, neighborhood sizes) on the order of hundreds of points, existing ID estimation methods may have only limited usefulness for applications in which the data consists of many natural groups of small size. In this paper, we propose a local ID estimation strategy stable even for ‘tight’ localities consisting of as few as 20 sample points. The estimator applies MLE techniques over all available pairwise distances among the members of the sample, based on a recent extreme-value-theoretic model of intrinsic dimensionality, the Local Intrinsic Dimension (LID). Our experimental results show that our proposed estimation technique can achieve notably smaller variance, while maintaining comparable levels of bias, at much smaller sample sizes than state-of-the-art estimators.

Current Research Information Systems (CRISs) offer great opportunities for assessments of institutional research outputs and extraction of useful and actionable knowledge based on various data-analysis techniques. However, many of these opportunities have not been explored in depth, especially in c ulture-sensitive areas such as gender-based analysis of research productivity and collaboration. In this paper we present GERBER, a network-based methodology and accompanying tool for gender-based analysis of publication data stored in institutional CRISs. GERBER relies on statistically robust techniques applied on weighted co-authorship networks whose nodes are enriched with different types of researcher evaluation metrics. The functionality of GERBER is demonstrated on publication data stored in the institutional CRIS of the Faculty of Sciences, University of Novi Sad, Serbia. The obtained results show that GERBER enables institutional research managers and policy makers to detect gender inequalities and homophily in research productivity and collaboration. Finally, we discuss different possibilities to integrate GERBER with CRISs in order to facilitate continuous gender-based evaluation of researchers.

Spatial centrality, whereby samples closer to the center of a dataset tend to be closer to all other samples, is regarded as one source of hubness. Hubness is well known to degrade k-nearest-neighbor (k-NN) classification. Spatial centrality can be removed by centering, i.e., shifting the origin to the global center of the dataset, in cases where inner product similarity is used. However, when Euclidean distance is used, centering has no effect on spatial centrality because the distance between the samples is the same before and after centering. As described in this paper, we propose a solution for the hubness problem when Euclidean distance is considered. We provide a theoretical explanation to demonstrate how the solution eliminates spatial centrality and reduces hubness. We then present some discussion of the reason the proposed solution works, from a viewpoint of density gradient, which is regarded as the origin of spatial centrality and hubness. We demonstrate that the solution corresponds to flattening the density gradient. Using real-world datasets, we demonstrate that the proposed method improves k-NN classification performance and outperforms an existing hub-reduction method.

Hubness has been recently identified as a problematic phenomenon occurring in high-dimensional space. In this paper, we address a different type of hubness that occurs when the number of samples is large. We investigate the difference between the hubness in high-dimensional data and the one in large-sample data. One finding is that centering, which is known to reduce the former, does not work for the latter. We then propose a new hub-reduction method, called localized centering. It is an extension of centering, yet works effectively for both types of hubness. Using real-world datasets consisting of a large number of documents, we demonstrate that the proposed method improves the accuracy of k-nearest neighbor classification.

Conceptual modelling education is often a part of any Informatics and Computer Science curricula but is also included in other study programs where conceptual modelling knowledge is needed. Conceptual modelling is well defined and numerous sources are available in existing literature. The steps to develop conceptual models are easy to learn and support is possible also via different tools, based on a theoretical background, mostly through the Peter Chen approach. In general, such an educational approach can be described as a standard conceptual modelling education. But, unfortunately, standard conceptual modeling education does not inform culturally-aware developers of conceptual models. The confidence and assurance that cross cultural issues and cultural awareness will be considered in the phase of conceptual modelling could be reached on one side by upgrading the conceptual modeling approach with a cultural point of view, as well as by introducing cultural issues into conceptual modelling education. Students need to be aware of basic cultural issues and concepts and their influence on conceptual modeling. In the long term, the mentioned content should become part of standard conceptual modeling education. This means changes in the content of curricula as well as a change in thinking by students and teachers.

Outlier detection in high-dimensional data presents various challenges resulting from the “curse of dimensionality.” A prevailing view is that distance concentration, i.e., the tendency of distances in high-dimensional data to become indiscernible, hinders the detection of outliers by making distance-based methods label all points as almost equally good outliers. In this paper, we provide evidence supporting the opinion that such a view is too simple, by demonstrating that distance-based methods can produce more contrasting outlier scores in high-dimensional settings. Furthermore, we show that high dimensionality can have a different impact, by reexamining the notion of reverse nearest neighbors in the unsupervised outlier-detection context. Namely, it was recently observed that the distribution of points' reverse-neighbor counts becomes skewed in high dimensions, resulting in the phenomenon known as hubness. We provide insight into how some points (antihubs) appear very infrequently in k-NN lists of other points, and explain the connection between antihubs, outliers, and existing unsupervised outlier-detection methods. By evaluating the classic k-NN method, the angle-based technique designed for high-dimensional data, the density-based local outlier factor and influenced outlierness methods, and antihub-based methods on various synthetic and real-world data sets, we offer novel insight into the usefulness of reverse neighbor counts in unsupervised outlier detection.

High-dimensional data arise naturally in many domains, and have regularly presented a great challenge for traditional data mining techniques, both in terms of effectiveness and efficiency. Clustering becomes difficult due to the increasing sparsity of such data, as well as the increasing difficulty in distinguishing distances between data points. In this paper, we take a novel perspective on the problem of clustering high-dimensional data. Instead of attempting to avoid the curse of dimensionality by observing a lower dimensional feature subspace, we embrace dimensionality by taking advantage of inherently high-dimensional phenomena. More specifically, we show that hubness, i.e., the tendency of high-dimensional data to contain points (hubs) that frequently occur in k-nearest-neighbor lists of other points, can be successfully exploited in clustering. We validate our hypothesis by demonstrating that hubness is a good measure of point centrality within a high-dimensional data cluster, and by proposing several hubness-based clustering algorithms, showing that major hubs can be used effectively as cluster prototypes or as guides during the search for centroid-based cluster configurations. Experimental results demonstrate good performance of our algorithms in multiple settings, particularly in the presence of large quantities of noise. The proposed methods are tailored mostly for detecting approximately hyperspherical clusters and need to be extended to properly handle clusters of arbitrary shapes.

Most machine-learning tasks, including classification, involve dealing with high-dimensional data. It was recently shown that the phenomenon of hubness, inherent to high-dimensional data, can be exploited to improve methods based on nearest neighbors (NNs). Hubness refers to the emergence of points (hubs) that appear among the k NNs of many other points in the data, and constitute influential points for kNN classification. In this paper, we present a new probabilistic approach to kNN classification, naive hubness Bayesian k-nearest neighbor (NHBNN), which employs hubness for computing class likelihood estimates. Experiments show that NHBNN compares favorably to different variants of the kNN classifier, including probabilistic kNN (PNN) which is often used as an underlying probabilistic framework for NN classification, signifying that NHBNN is a promising alternative framework for developing probabilistic NN algorithms.

In the context of many data mining tasks, high dimensionality was shown to be able to pose significant problems, commonly referred to as different aspects of the curse of dimensionality. In this paper, we investigate in the time-series domain one aspect of the dimensionality curse called hubness, which refers to the tendency of some instances in a data set to become hubs by being included in unexpectedly many k-nearest neighbor lists of other instances. Through empirical measurements on a large collection of time-series data sets we demonstrate that the hubness phenomenon is caused by high intrinsic dimensionality of time-series data, and shed light on the mechanism through which hubs emerge, focusing on the popular and successful dynamic time warping (DTW) distance. Also, the interaction between hubness and the information provided by class labels is investigated, by considering label matches and mismatches between neighboring time series. Following our findings we formulate a framework for categorizing time-series data sets based on measurements that reflect hubness and the diversity of class labels among nearest neighbors. The framework allows one to assess whether hubness can be successfully used to improve the performance of k-NN classification. Finally, the merits of the framework are demonstrated through experimental evaluation of 1-NN and k-NN classifiers, including a proposed weighting scheme that is designed to make use of hubness information. Our experimental results show that the examined framework, in the majority of cases, is able to correctly reflect the circumstances in which hubness information can effectively be employed in k-NN time-series classification.

The vector space model (VSM) is a popular and widely applied model in information retrieval (IR). VSM creates vector spaces whose dimensionality is usually high (e.g., tens of thousands of terms). This may cause various problems, such as susceptibility to noise and difficulty in capturing the underlying semantic structure, which are commonly recognized as different aspects of the "curse of dimensionality." In this paper, we investigate a novel aspect of the dimensionality curse, which is referred to as hubness and manifested by the tendency of some documents (called hubs) to be included in unexpectedly many search result lists. Hubness may impact VSM considerably since hubs can become obstinate results, irrelevant to a large number of queries, thus harming the performance of an IR system and the experience of its users. We analyze the origins of hubness, showing it is primarily a consequence of high (intrinsic) dimensionality of data, and not a result of other factors such as sparsity and skewness of the distribution of term frequencies. We describe the mechanisms through which hubness emerges by exploring the behavior of similarity measures in high-dimensional vector spaces. Our consideration begins with the classical VSM (tf-idf term weighting and cosine similarity), but the conclusions generalize to more advanced variations, such as Okapi BM25. Moreover, we explain why hubness may not be easily mitigated by dimensionality reduction, and propose a similarity adjustment scheme that takes into account the existence of hubs. Experimental results over real data indicate that significant improvement can be obtained through consideration of hubness.

High dimensionality can pose severe difficulties, widely recognized as different aspects of the curse of dimensionality. In this paper we study a new aspect of the curse pertaining to the distribution of k-occurrences, i.e., the number of times a point appears among the k nearest neighbors of other points in a data set. We show that, as dimensionality increases, this distribution becomes considerably skewed and hub points emerge (points with very high k-occurrences). We examine the origin of this phenomenon, showing that it is an inherent property of high-dimensional vector space, and explore its influence on applications based on measuring distances in vector spaces, notably classification, clustering, and information retrieval.

A crucial operation in memory-based collaborative filtering (CF) is determining nearest neighbors (NNs) of users/items. This paper addresses two phenomena that emerge when CF algorithms perform NN search in high-dimensional spaces that are typical in CF applications. The first is similarity concentration and the second is the appearance of hubs (i.e. points which appear in $k$-NN lists of many other points). Through theoretical analysis and experimental evaluation we show that these phenomena are inherent properties of high-dimensional space, unrelated to other data properties like sparsity, and that they can impact CF algorithms by questioning the meaning and representativeness of discovered NNs. Moreover, we show that it is not easy to mitigate the phenomena using dimensionality reduction. Studying these phenomena aims to provide a better understanding of the limitations of memory-based CF and motivate the development of new algorithms that would overcome them.