Bidirectional Transformer architectures are state-of-the-art sequential recommendation models that use a bi-directional representation capacity based on the Cloze task, a.k.a. Masked Language Modeling. The latter aims to predict randomly masked items within the sequence. Because they assume that the true interacted item is the most relevant one, an exposure bias results, where non-interacted items with low exposure propensities are assumed to be irrelevant. The most common approach to mitigating exposure bias in recommendation has been Inverse Propensity Scoring (IPS), which consists of down-weighting the interacted predictions in the loss function in proportion to their propensities of exposure, yielding a theoretically unbiased learning. In this work, we argue and prove that IPS does not extend to sequential recommendation because it fails to account for the temporal nature of the problem. We then propose a novel propensity scoring mechanism, which can theoretically debias the Cloze task in sequential recommendation. Finally we empirically demonstrate the debiasing capabilities of our proposed approach and its robustness to the severity of exposure bias.

Recent work in recommender systems has emphasized the importance of fairness, with a particular interest in bias and transparency, in addition to predictive accuracy. In this paper, we focus on the state of the art pairwise ranking model, Bayesian Personalized Ranking (BPR), which has previously been found to outperform pointwise models in predictive accuracy, while also being able to handle implicit feedback. Specifically, we address two limitations of BPR: (1) BPR is a black box model that does not explain its outputs, thus limiting the user’s trust in the recommendations, and the analyst’s ability to scrutinize a model’s outputs; and (2) BPR is vulnerable to exposure bias due to the data being Missing Not At Random (MNAR). This exposure bias usually translates into an unfairness against the least popular items because they risk being under-exposed by the recommender system. In this work, we first propose a novel explainable loss function and a corresponding Matrix Factorization-based model called Explainable Bayesian Personalized Ranking (EBPR) that generates recommendations along with item-based explanations. Then, we theoretically quantify additional exposure bias resulting from the explainability, and use it as a basis to propose an unbiased estimator for the ideal EBPR loss. The result is a ranking model that aptly captures both debiased and explainable user preferences. Finally, we perform an empirical study on three real-world datasets that demonstrate the advantages of our proposed models.

The closed feedback loop in recommender systems is a common setting that can lead to different types of biases. Several studies have dealt with these biases by designing methods to mitigate their effect on the recommendations. However, most existing studies do not consider the iterative behavior of the system where the closed feedback loop plays a crucial role in incorporating different biases into several parts of the recommendation steps.

Real-time micro-blogging services such as Twitter generate billions of micro-posts to share information, daily. However, these posts are inherently noisy and unstructured, hence making them difficult to organize for the purpose of retrieval of relevant information. Hashtags are quickly becoming the standard approach for annotation of various information on social media. However hashtags are not used in a consistent manner and most importantly, are completely optional to use. This makes them unreliable as the sole mechanism for searching for relevant information. In this paper, we investigate mechanisms for consolidating the hashtag space using recom-mender systems. We propose a cross-domain collaborative filtering recommender system based on matrix factorization (MF) combined with a hashtag similarity graph, to automatically predict relevant hashtags based on an analysis of existing data and the relations among hashtags. Our experiments confirm that our approach outperforms several baselines. Our research using these hashtag graphs for recommendations has further revealed a promising mechanism for revealing stories from the cliques in this graph, paving the road toward story annotation as an extension of hashtag recommendation for tweets.

Personalized recommender systems are becoming increasingly relevant and important in the study of polarization and bias, given their widespread use in filtering information spaces. Polarization is a social phenomenon, with serious consequences, in real-life, particularly on social media. Thus it is important to understand how machine learning algorithms, especially recommender systems, behave in polarized environments. In this paper, we study polarization within the context of the users' interactions with a space of items and how this affects recommender systems. We first formalize the concept of polarization based on item ratings and then relate it to the item reviews to investigate any potential correlation. We then propose a domain independent data science pipeline to automatically detect polarization using the ratings rather than the typical properties used to detect polarization, such as item's content or social network topology. We perform an extensive comparison of polarization measures on several benchmark data sets and show that our polarization detection framework can detect different degrees of polarization and outperforms existing measures in capturing an intuitive notion of polarization. Our work is an essential step toward quantifying and detecting polarization in ongoing ratings and in benchmark data sets, and to this end, we use our developed polarization detection pipeline to compute the polarization prevalence of several benchmark data sets. It is our hope that this work will contribute to supporting future research in the emerging topic of designing and studying the behavior of recommender systems in polarized environments.

Accurate model-based Collaborative Filtering (CF) approaches, such as Matrix Factorization (MF), tend to be black-box machine learning models that lack interpretability and do not provide a straightforward explanation for their outputs. Yet explanations have been shown to improve the transparency of a recommender system by justifying recommendations, and this in turn can enhance the user's trust in the recommendations. Hence, one main challenge in designing a recommender system is mitigating the trade-off between an explainable technique with moderate prediction accuracy and a more accurate technique with no explainable recommendations. In this paper, we focus on factorization models and further assume the absence of any additional data source, such as item content or user attributes. We propose an explainability constrained MF technique that computes the top-n recommendation list from items that are explainable. Experimental results show that our method is effective in generating accurate and explainable recommendations.

Glaucoma is a silent eye disease that steals sight without warning and it is the second leading cause of blindness worldwide. It cannot be cured but it can be controlled. A gene network analysis to understand the cross talk between glaucoma networks of genes could contribute to better understanding and treatment of the disease. In this paper, we extend our previous text mining and network analysis research for extraction of a more comprehensive glaucoma interaction network from PubMed Central articles. After text mining of the PubMed Central literature to extract potential sentences bearing associations between glaucoma genes in the literature, the mined associations are used to construct a glaucoma gene interaction network and a network analysis is applied to understand the building structure of the network. The network analysis reveals a succinct network composed of five distinct clusters associated with seven benchmark glaucoma biomarkers. The knowledge that can be extracted from such a network could provide a useful summative knowledge base to complement other forms of clinical information related to this disease that affects many diagnosed individuals worldwide.

Topic Models are statistical models that can be used for discovering the abstract “topics” that may occur in a text corpus, however they face dramatic challenges when coping with very sparse and yet topically diverse micro-blog posts such as tweets. In such streams, not only are the topics very diverse, but also the vocabulary is huge, making the sampling space for generative models vast. In this work, we propose a topic modeling framework built on top of an existing stream clustering framework, called Stream Dashboard, which can extract, isolate, and track evolving topics over any given time period. The Stream Dashboard module initially clusters the data stream points into homogeneous groups which are then subjected to online topic modeling, thus making our approach a compartmentalized topic modeling approach. The proposed evolving topic mining framework can handle the noise, size, dynamic nature, and diversity of the data stream by partitioning the data in both the content and time spaces, hence making the challenging topic modeling task easier and faster and making the cluster models resulting from Stream-Dashboard compatible with topic modeling, thus more appropriate for text data.

We bring to the fore of the recommender system research community, an inconvenient truth about the current state of understanding how recommender system algorithms and humans influence one another, both computationally and cognitively. Unlike the great variety of supervised machine learning algorithms which traditionally rely on expert input labels and are typically used for decision making by an expert, recommender systems specifically rely on data input from non-expert or casual users and are meant to be used directly by these same non-expert users on an every day basis. Furthermore, the advances in online machine learning, data generation, and predictive model learning have become increasingly interdependent, such that each one feeds on the other in an iterative cycle. Research in psychology suggests that people's choices are (1) contextually dependent, and (2) dependent on interaction history. Thus, while standard methods of training and assessing performance of recommender systems rely on benchmark datasets, we suggest that a critical step in the evolution of recommender systems is the development of benchmark models of human behavior that capture contextual and dynamic aspects of human behavior. It is important to emphasize that even extensive real life user-tests may not be sufficient to make up for this gap in benchmarking validity because user tests are typically done with either a focus on user satisfaction or engagement (clicks, sales, likes, etc) with whatever the recommender algorithm suggests to the user, and thus ignore the human cognitive aspect. We conclude by highlighting the interdisciplinary implications of this endeavor.

We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (nonnegativity-constrained autoencoder), that learns features that show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into parts and its prediction performance is tested on three standard image data sets and one text data set. The results indicate that the nonnegativity constraint forces the autoencoder to learn features that amount to a part-based representation of data, while improving sparsity and reconstruction quality in comparison with the traditional sparse autoencoder and nonnegative matrix factorization. It is also shown that this newly acquired representation improves the prediction performance of a deep neural network.

In this paper, we describe a fully automatic learner modeling approach in learning management systems, taking into account learners' educational preferences including learning styles. We propose a learner model with three components: the learner's profile, learner's knowledge, and learner's educational preferences. The learner's profile represents the learner's general information such as identification data, the learner's knowledge implies the learner's interests on visited learning objects, and the learner's educational preferences are composed of the learner's preferences among visited learning objects and his/her learning style. In the proposed approach, all learner model components are automatically detected, without requiring explicit feedback. Indeed, all the basic learners' information is inferred from the learners' online activities and usage data, based on web usage mining techniques and a literature-based approach for the automatic detection of learning styles in learning management systems. Once learner models are built, we apply a hierarchical multi-level model based collaborative filtering approach, in order to gather learners with similar preferences and interests in the same groups.

The identification of community structure is one of the fundamental questions in the analysis of large scale complex networks. In this work, we propose a novel approach to extracting communities within a large network of cyber learners and learning resources. The technique used is a heuristic which initially performs clustering using force-based visualization algorithms and then relies on network modularity to select good decompositions from those found visually. Through testing, we have determined appropriate parameters for optimal performance. Finally, we use the community detection method to design a visual recommender system to recommend learning resources to cyber learners within the same community.

We propose a method to extract salient contour groups from cluttered regions using Markov Random Fields. Our technique delineates smooth, long, and elliptical curves out of clutter. To extract salient curves, we use the following perceptual rules: smoothness, proximity, co-circularity, co-elliptic, and length. Our method consists of the following steps: obtaining individual smooth contours along their saliency measures; starting from the most salient contour search for possible grouping options for each contour; continuing the grouping until an optimum solution is reached. We introduce circularity and saliency measures for open curves. We applied our method to discern coronal loops from cluttered solar images, and were able to successfully obtain the entire set of desired coronal loops while eliminating any cluttered background.

One of the first Multi-Language Information Retrieval (MLIR) systems was implemented in 1969 by Gerard Salton who enhanced his SMART system to retrieve multilingual documents in two languages, English and German. However, the research field of MLIR is still struggling since the majority of information retrieval systems are monolingual and more precisely English-based, even though only 6% of the world's population native language have as English [14]. This paper presents a Multi-Language Information Retrieval (MLIR) approach that falls into the area of Domain Specific Information Retrieval (E-learning being the domain). The approach we followed is a synergistic approach between (1) Thesaurus-based Approach and (2) Corpus-based Approach. This research has been implemented on a real platform called HyperManyMedia1at Western Kentucky University.

In this paper, we present a scalable evolutionary algorithm for clustering large and dynamic data sets, called Scalable Evolutionary Clustering with Self Adaptive Genetic Operators (Scalable ECSAGO). The proposed evolutionary clustering algorithm can adapt its genetic operators rate while the evolution leads to the optimal centers of the clusters. The sizes of the clusters are estimated using a hybrid analytical optimization procedure. Moreover, a memorization factor is introduced in order to allow the algorithm to keep as much of the previously discovered knowledge about clusters and data summarization as desired. The proposed scalable ECSAGO algorithm is able to find accurate representations of the clusters on very large data sets of different sizes and dimensionality that might not fit in main memory, while maintaining the desirable properties of robustness to noise and automatic detection of the number of clusters. The algorithm is also useful for traking evolving cluster structures that change with the passage of time.

This paper proposes a swarm intelligence based recommender system (FlockRecom) based on the collaborative behavior of bird flocks for generating Top-N recommendations. The flock-based recommender algorithm (FlockRecom) iteratively adjusts the position and speed of dynamic flocks of agents on a visua

The main objective of this paper is to propose and evaluate an architecture that provides, manages, and collects data that permit high levels of adaptability and relevance to the user profiles. In addition, we implement this architecture on a platform called HyperManyMedia. To achieve this objective, an approach for personalized search is implemented that takes advantage of the semantic Web standards (RDF and OWL) to represent the content and the user profiles. The framework consists of the following phases: (1) building the semantic E-learning domain using the known college and course information as concept and sub-concept, (2) generating the semantic user profiles as ontologies, (3) clustering the documents to discover more refined sub-concepts, (4) re-ranking the user's search results based on his/her profile, and (5) providing the user with semantic recommendations. The implementation of the ontologies models is separate from the design and implementation of the information retrieval system, thus providing a modular framework that is easy to adapt and port to other platforms. Finally, the experimental results show that the user context can be effectively used for improving the precision and recall in E-learning search, particularly by re-ranking the search results based on the user profiles.

In this paper, we describe a system that determines coronal loop existence from a given Solar image region in two stages: 1) extracting principal contours from the solar image regions, 2) deciding whether the extracted contours are in a loop shape. In the first stage, we propose a principal contour extraction method that achieves 88% accuracy in extracting the desired contours from the cluttered regions. In the second stage, we analyze the extracted contours in terms of their geometric features such as linearity, elliptical features, curvature, proximity, smoothness, and corner points. To distinguish loop contours from the other forms, we train an Adaboost classifier based C4.5 decision tree by using geometric features of 150 loop contours and 250 non-loop contours. Our system achieves 85% F1-Score from 10-fold cross validation experiments.

Visual content has become an important component of the web. In many cases, visual content is mixed with other modalities (e.g. text) that can be exploited to extract information and knowledge. This paper presents a strategy for mining multimodal visual content. The strategy encompasses two main components: a rich representation of the multimodal objects and a model for automatically annotating unannotated images. The proposed method has two distinguishing characteristics: it uses a bag-of-features representation for images and a non-negative matrix factorization algorithm to build a latent representation.

Designing personalized search engines based on a recommender system that takes into consideration the user situated moment in relation to the subject matter and the context that governs user interest has been largely ignored. In this paper, we present a novel approach to integrating user interests into search within a recommender system that is guided by the semantic representation of the user and the content. In addition, our research tackles two problems of creating any recommender system: (a) the initial stage problem (how to provide recommendations to a user if the system hasn't been used yet) and (b) user context (providing the same user with different recommendations based on the context of their recent activity). Also the design of our recommender system is modular. It integrates and accommodates user's preferences by using User Relevance Feedback. Finally, we describe how the personalization aspects can increase the recommendation quality.

In order to automatically provide the most appropriate learning objects to e-learners, a special interest should be given to the process of building the learnerspsila models. First, we need to identify which relevant information to include in the learnerpsilas model, while taking into account various pedagogical considerations, and second how to accurately infer the learnerspsila preferences and characteristics from their online behavior and activities. This paper presents a preliminary study of the possibility of integrating educational preferences in the learnerpsilas model and detecting them automatically within e-learning systems.

This paper presents a framework for identifying the legal risks associated with performing network forensics on public networks. The framework is discussed in the context of the Gnutella P2P network protocol for which the legal issues related to authorized access have not yet been addressed.

The Semantic Web has been evolved as a key technology, bringing new insights and solutions to well documented problems in E-learning. The research field of Semantic E-learning covers a wide range of research problems. In this paper, we present an approach to visualize an ontology driven information retrieval system in the context of E-learning. This approach has been implemented on the HyperManyMedia platform, and is already being used by online students at WKU. The design of this work is based on the following steps: (1) presenting the HyperManyMedia domain as a tree consisting of concepts and subconcepts where the leaves are the documents (lectures), (2) embedding Nutch search engine into the platform, (3) reconfiguring Nutch search engine to accommodate the semantic search, (4) using Prefuse as a visual tool to present the ontology as a graph, and (5) embedding the graph into the platform as an applet so that learners can navigate visually through the ontology of the domain by clicking on nodes.

We present a generic framework to evaluate patterns obtained from transactional web data streams whose underlying distribution changes with time. The evolving nature of the data makes it very difficult to determine whether there is structure in the data stream, and whether this structure is being learned. This challenge arises in applications such as mining online store transactions, summarizing dynamic document collections, and profiling web traffic. We propose to evaluate this hard instance of unsupervised learning using a continuous assessment of the predictive power of the learned patterns, with specific examples that borrow concepts from supervised learning. We present results from experiments with synthetic data, the 20 Newsgroups dataset, web clickstream data, and a custom collection of RSS News feeds.

Recent years have transformed the Web from a Web of content to a Web of applications and social content. Thus, it has become crucial to be able to tap on this social aspect of the Web whenever possible, in addition to its content, particularly for focused crawling. In this paper, we present a novel profile-based focused crawling system for dealing with the increasingly popular social media-sharing web sites without assuming any privileged access to the internal private databases of such websites, nor any requirement for the existence of APIs for the extraction of social data. Our experiments prove the robustness of our profile-based focused crawler, as well as a significant improvement in harvest ratio, compared to breadth-first and OPIC crawlers, when crawling the flickr web site for two different topics.

The World Wide Web (WWW) is becoming one of the most preferred and widespread mediums of learning. Unfortunately, most of the current Web-based learning systems are still delivering the same educational resources in the same way to learners with different profiles. A number of past efforts have dealt with e-learning personalization, generally, relying on explicit information. In this paper, we aim to compute on-line automatic recommendations to an active learner based on his/her recent navigation history, as well as exploiting similarities and dissimilarities among user preferences and among the contents of the learning resources. First we start by mining learner profiles using Web usage mining techniques and content-based profiles using information retrieval techniques. Then, we use these profiles to compute relevant links to recommend for an active learner by applying a number of different recommendation strategies.

Inspired by the success of inverted indexing in the textual search domain, we provide sparseness justifications for using inverted file indexing on image content, which paves the way for developing scalable image content search systems. We use clustering to automatically generate a content vocabulary. To avoid the problem of generating cluster centers that are overcrowded in high density areas for sparse data sets, we use a cluster-merge procedure for cluster post-processing. We further use visual codewords to represent low level image features, which not only makes the inverted file indexing and search applicable to image content, but also helps bridge the gap between the low level image features and high-level human visual perception. Experimental results confirm the success of our methods.

Coronal loops are especially important in analyzing some important phenomena related to the Sun such as the controversial coronal heating problem. The analysis requires astrophysicists to manually sift through thousands of images in order to acquire images containing coronal loops. Thus, the motivation to detect these loops automatically. Since coronal loops do not have a perfect shape and are easy to confuse with other solar events, feature selection to learn characteristics of loops requires special care. In this study, we explore standard image features as well as specialized image features considering coronal loop characteristics. Our experiments confirm the success of our explored features in coronal loop detection.

In this paper, we use a flock of agent-based swarm intelligence approach for simultaneously clustering and visualizing high-dimensional Web usage data. Our approach is based on improvements that overcome several limitations of the FClust algorithm. Our proposed approach is a hybrid, combining the strengths of the spherical k-means algorithm for fast clustering of high-dimensional data sets in the original feature domain and the flock-based (FClust) algorithm which iteratively adjusts the position and speed of dynamic flocks of agents on a visualization plane. Our hybridization decreases the complexity of FClust from quadratic to linear, with further improvements in the cluster quality. Experiments on real data illustrate the workings of the hybrid algorithm and its advantages over its FClust baseline.

In this paper, we present a purely incremental, scalable algorithm for the detection of elliptical shapes in images. Our method uses an incremental version of the Random Hough Transform (RHT) to compute the curve parameters from sampled image points and uses a density-based robust stream clustering algorithm to discover the potential parameters from the Hough space. Finally we apply density and similarity tests to eliminate weak and redundant candidates. Being totally incremental, and not requiring the typically huge memory costs of Hough accumulator arrays or image pixels, our method reduces the number of computations performed and the memory used. The proposed method is tested on both synthetic and real images, including solar images captured by various instruments onboard NASA and ESA satellites.

In this paper, we present a novel profile based focused crawling system for dealing with increasingly popular social media-sharing Web sites. In this system, we treat users' profiles as ranking criteria for guiding the crawling process. Furthermore, we divide a user's profile into two parts, an internal part, which comes from the user's own contribution, and an external part, which comes from the user's social contacts. In order to efficiently and effectively extract data from a social media-sharing website for focused crawling, a path string based page-classification method was first developed for identifying list pages, detail pages and profile pages.

We present an approach for personalized retrieval in an e-learning platform, that takes advantage of semantic Web standards to represent the learning content and the user/learner profiles as ontologies, and that re-ranks search results/lectures based on how the contained terms map to these ontologies. One important aspect of our approach is the combination of an authoritatively supplied taxonomy by the colleges, with the data driven extraction (via clustering) of a taxonomy from the documents themselves, thus making it easier to adapt to different learning platforms, and making it easier to evolve with the document/lecture collection. Our experimental results show that the learner's context can be effectively used for improving the precision and recall in e-learning content retrieval, particularly by re-ranking the search results based on the learner's past activities.

In this paper, we present a complete framework and findings in mining Web usage patterns from Web log files of a real Web site that has all the challenging aspects of real-life Web usage mining, including evolving user profiles and external data describing an ontology of the Web content. Even though the Web site under study is part of a nonprofit organization that does not "sell" any products, it was crucial to understand "who" the users were, "what" they looked at, and "how their interests changed with time," all of which are important questions in Customer Relationship Management (CRM). Hence, we present an approach for discovering and tracking evolving user profiles. We also describe how the discovered user profiles can be enriched with explicit information need that is inferred from search queries extracted from Web log data. Profiles are also enriched with other domain-specific information facets that give a panoramic view of the discovered mass usage modes. An objective validation strategy is also used to assess the quality of the mined profiles, in particular their adaptability in the face of evolving user behavior.

In this paper, we study the behavior of collaborative filtering based recommendations under evolving user profile scenarios. We propose a systematic validation methodology that allows for simulating various controlled user profile evolution scenarios and validating the studied recommendation strategies. Through the presented work, we observe the effect of the curse of dimensionality and sparsity that can wreck havoc on collaborative filtering in a streaming scenario, and conclude that a hybrid approach with both content and collaborative filtering may be the way to go in a high sparsity streaming scenario.

In stream data mining it is important to use the most recent data to cope with the evolving nature of the underlying patterns. Simply keeping the most recent records offers no flexibility about which data is kept, and does not exploit even minimal redundancies in the data (a first step towards pattern discovery). This paper focuses in how to construct and maintain efficiently (in one pass) a compact summary for data such as web logs and text streams. The resulting structure is a prefix tree, with ordering criterion that changes with time, such as an activity time stamp or attribute frequency. A detailed analysis of the factors that affect its performance is carried out, including empirical evaluations using the well known 20 Newsgroups data set. Guidelines for forgetting and tree pruning are also provided. Finally, we use this data structure to discover evolving topics from the 20 Newsgroups.

In this paper, we present novel methods that combine (1) Markov models and (2) Web page content search techniques to generate Web navigation recommendations. For click-stream modeling, both first-order and second-order Markov models were studied and a compact storage format for Markov transition matrices was used. For content-based search, a search engine was used to obtain similar-content pages for recommendation to compensate for the sparsity of the Markov model and thus improve coverage. Experiments were conducted on real Web clickstream logs, and confirmed the efficiency of the proposed methods.

We present an algorithm for Evolutionary Clustering with Self Adaptive Genetic Operators (ECSAGO). This algorithm is based on the Unsupervised Niche Clustering (UNC) and Hybrid Adaptive Evolutionary (HAEA) algorithms. The UNC is a genetic clustering algorithm that is robust to noise and is able to determine the number of clusters automatically. HAEA is a parameter adaptation technique that automatically learns the rates of its genetic operators at the same time that the individuals are evolved in an Evolutionary Algorithm (EA). ECSAGO uses an EA with real encoding, real genetic operators, and adapts the genetic operator rates as it is evolving the cluster prototypes. This will have the advantage of reducing the number of parameters required by UNC (thus avoiding the problem of fixing the genetic operator parameter values), and solving problems where real representation is required or prefered for the solutions.

This paper presents a simple and intuitive method for mining search engine query logs to get fast query recommendations on a large scale industrial-strength search engine. In order to get a comprehensive solution, we combine two methods together. First, we study and model search engine users' sequential search behavior, and interpret this consecutive search behavior as client-side query refinement, that should form the basis for the search engine's own query refinement process. This query refinement process is exploited to learn useful relations and build fuzzy associative memories that help generate related queries via a fuzzy inference process. Second, we combine this method with a traditional content based similarity method to compensate for the high sparsity of real query log data, and more specifically, the shortness of most query sessions.

We cast the frequent itemset mining problem as a criterion guided optimization problem instead of one based on exact counting. This opens several interesting possibilities, including modification of the criterion function to take into account (i) error tolerance, (ii) locality, (iii) unsupervised estimation of the error tolerance, and (iv) search strategy. We also propose a new validation procedure that takes into account the completeness and accuracy of the discovered patterns. Experiments with real Web transaction data are presented.

We present a new approach for tracking evolving and noisy data streams by estimating clusters based on density, while taking into account the possibility of the presence of an unknown amount of outliers, the emergence of new patterns, and the forgetting of old patterns.

No abstract available.

This paper presents a simple and intuitive method for mining search engine query logs to get fast query recommendations on a large scale industrial strength search engine. In order to get a more comprehensive solution, we combine two methods together. On the one hand, we study and model search engine users' sequential search behavior, and interpret this consecutive search behavior as client-side query refinement, that should form the basis for the search engine's own query refinement process. On the other hand, we combine this method with a traditional content based similarity method to compensate for the high sparsity of real query log data, and more specifically, the shortness of most query sessions. To evaluate our method, we use one hundred day worth query logs from SINA' search engine to do off-line mining. Then we analyze three independent editors evaluations on a query test set. Based on their judgement, our method was found to be effective for finding related queries, despite its simplicity. In addition to the subjective editors' rating, we also perform tests based on actual anonymous user search sessions.

We present a systematic approach to automatic Web recommender systems based on Web usage mining in a first stage to learn user profiles, and a second data mining phase that is devoted to learning several accurate models for predicting user requests for each profile. Our approach differs from existing methods because it includes two separate learning phases: one to learn the user profiles, and another to learn a recommendation model. Most previous approaches do not include adaptive learning in a separate second phase, and instead base the recommendations on simple assumptions such as nearest profile recommendations, or deployment of pre-discovered association rules

While scalable data mining methods are expected to cope with massive Web data, coping with evolving trends in noisy data in a continuous fashion, and without any unnecessary stoppages and reconfigurations is still an open challenge. This dynamic and single pass setting can be cast within the framework of mining evolving data streams. In this paper, we explore the task of mining mass user profiles by discovering evolving Web session clusters in a single pass with a recently proposed scalable immune based clustering approach (TECNO-STREAMS), and study the effect of the choice of different similarity measures on the mining process and on the interpretation of the mined patterns. We propose a simple similarity measure that has the advantage of explicitly coupling the precision and coverage criteria to the early learning stages, and furthermore requiring that the affinity of the data to the learned profiles or summaries be defined by the minimum of their coverage or precision, hence requiring that the learned profiles are simultaneously precise and complete, with no compromises.In our experiments, we study the task of mining evolving user profiles from Web clickstream data (web usage mining) in a single pass, and under different trend sequencing scenarios, showing that compared oto the cosine similarity measure, the proposed similarity measure explicitly based on precision and coverage allows the discovery of more correct profiles at the same precision or recall quality levels.

We present a new approach to anomaly detection based on unsupervised niche clustering (UNC). The UNC is a genetic niching technique for clustering that can handle noise, and is able to determine the number of clusters automatically. The UNC uses the normal samples for generating a profile of the normal space (clusters). Each cluster can later be characterized by a fuzzy membership function that follows a Gaussian shape defined by the evolved cluster centers and radii. The set of memberships are aggregated using a max-or fuzzy operator in order to determine the normalcy level of a data sample. Experiments on synthetic and real data sets, including a network intrusion detection data set, are performed and some results are analyzed and reported.

This paper introduces a generalization of the Gravitational Clustering Algorithm. First, it is extended in such a way that the Gravitational Law is not the only law that can be applied. Instead, any decreasing function of the distance between points can be used. An estimate of the maximum distance between the closest points is calculated in order to reduce the sensibility of the clustering process to the size of the data set. Finally, a heuristic for setting the interaction strength (gravitational constant) is introduced in order to reduce the number of parameters of the algorithm. Experiments with benchmark synthetic data sets are performed in order to show the applicability of the proposed approach.

Intelligent Web personalization aims at adapting a user's interaction with the Web information space based on information gathered about the user. A complete automated Web personalization system is generally based on Web usage mining to discover useful knowledge about user access patterns, followed by a recommendation system to act on this knowledge in order to respond to the users' individual interest, in a manner transparent to the user, and while protecting the user's privacy and anonymity. The flow of information in a Web personalization system can be prone to significant amounts of error and uncertainty. This uncertainty pervades all stages from the user's Web navigation patterns to the final recommendations, including the intermediate stages of logging Web usage, preprocessing and segmenting Web log data into Web user sessions, clustering these sessions, and computing Web user profiles from these clusters. Fuzzy approximate reasoning can offer a general framework for the, recommendation process. It is this framework that is investigated in this paper. This paper presents a simple, intuitive, and fast approach to provide dynamic predictions in the Web navigation space. Real Web usage data is used as a simulation testbed for the fuzzy approximate reasoning based recommendation system.

Many real world problems are dynamic in nature, and they deal with changing environments or objective functions. Dynamic objective functions can make the evolutionary search tedious or unsuccessful for Genetic Algorithms. Some work has focused on altering the evolutionary process, including the selection strategy, genetic operators, replacement strategy, or fitness modification. While other work focused on the concept of genotype to phenotype mapping or gene expression. This line of work includes models based on diploidy and dominance, messy GAs, proportional GA, overlapping genes such as in DNA coding method, the floating point representation, and the structured GA. In particular, the structured GA uses a simple structured hierarchical chromosome representation, where lower level genes are collectively switched on or off by specific higher level genes. Genes that are switched on are expressed into the final phenotype, while genes that are switched off do not contribute to coding the phenotype. We have recently proposed a modification of the sGA based on the concept of soft activation mechanism. The lower level genes are no longer limited to total expression or to none. Instead, they can be expressed to different gradual degrees. The soft structured Genetic Algorithm (s/sup 2/GA) inherits all the advantages of its crisp (non-fuzzy) counterpart (sGA), and possesses several additional unique features compared to the sGA and other GA based techniques. In this paper, we empirically demonstrate several strengths of the S/sup 2/GA approach with regard to non-stationary objective function optimization.

Artificial immune system (AIS) models hold many promises in the field of unsupervised learning. However, existing models are not scalable, which makes them of limited use in data mining. We propose a new AIS based clustering approach (TECNO-STREAMS) that addresses the weaknesses of current AIS models. Compared to existing AIS based techniques, our approach exhibits superior learning abilities, while at the same time, requiring low memory and computational costs. Like the natural immune system, the strongest advantage of immune based learning compared to other approaches is expected to be its ease of adaptation to the dynamic environment that characterizes several applications, particularly in mining data streams. We illustrate the ability of the proposed approach in detecting clusters in noisy data sets, and in mining evolving user profiles from Web clickstream data in a single pass. TECNO-STREAMS adheres to all the requirements of clustering data streams: compactness of representation, fast incremental processing of new data points, and clear and fast identification of outliers.

This paper presents a novel unsupervised robust clustering technique based on the Gravitational Law and the second Newton's motion Law. The technique automatically determines the number of clusters in the target data set. Basically, each data record in the source data set is considered as an object in the feature space. The objects are moved by using the gravitational force and the second motion law. Because the clusters define a disjoint collection of sets, an optimal disjoint set union-find structure is used to store and update the clusters that are being conformed by closest objects. The proposed technique is robust to noise, can be used to generate a partition of the data set at multiple resolution levels, and can also be used to extract seeds to form a good summary of the data. Experiments with synthetic and real data were conducted to show the performance of the proposed clustering technique.

We propose an approach to clustering text documents based on a coupled process of clustering and cluster-dependent keyword weighting. The proposed approach is based on the the fuzzy c-means clustering algorithm. Hence it is computationally and implementationally simple. Moreover, it learns a different set of keyword weights for each cluster. This means that, as a by-product of the clustering process, each document cluster will be characterized by a possibly different set of keywords. The cluster dependent keyword weights help in partitioning the document collection into more meaningful categories. They can also be used to automatically generate a brief summary of each cluster in terms of not only the attribute values, but also their relevance. For the case of text data, this approach can be used to automatically annotate the documents. We illustrate the performance of the proposed algorithm by using it to cluster a real collection of text documents.

The human immune system can be seen as a complex network structure that is able to respond to an almost unlimited multitude of foreign invaders such as viruses and bacteria. Hence, this parallel and distributed adaptive system promises tremendous potential in many intelligent computing applications, including Web mining. Some of these immunity-based techniques involve the development and analysis of algorithms that can identify patterns in observed data in order to make predictions about unseen data. In this paper, we introduce several new enhancements to deal with some of the weaknesses of previous artificial immune system models. In particular, we address the uncertainty and fuzziness inherent in the matching process that takes place between antibodies and antigens. This problem is handled by introducing a fuzzy artificial immune system. A fuzzy artificial immune system mimicking the body's adaptive learning and defense mechanism in the face of invading biological agents is used as a monitoring and learning system for a Web site in the face of all incoming Web requests.

% & ' )( * + , -( .(/ 10 % 2 3 4 ( 5 '6 7 8 9 : ; 2 < = > ( ?

We present a technique for simultaneously mining Web navigation patterns and maximally frequent context-sensitive itemsets (URL associations) from the historic user access data stored in Web server logs. A new hierarchical clustering technique that exploits the symbiosis between clusters in feature space and genetic biological niches in nature, called Hierarchical Unsupervised Niche Clustering (H-UNC) is presented. We use H-UNC as part of a complete system of knowledge discovery in Web usage data. Our approach does not necessitate fixing the number of clusters in advance, is insensitive to initialization, can handle noisy data, general non-differentiable similarity measures, and automatically provides profiles at multiple resolution levels. Our experiments show that our algorithm is not only capable of extracting meaningful user profiles on real Web sites, but also discovers associations between distinct URL pages on a site, with no additional cost. Unlike content based association methods, our approach discovers associations between different Web pages based only on the user access patterns and not on the page content. Also, unlike traditional context-blind association discovery methods, H-UNC discovers context-sensitive associations which are only meaningful within a limited context/user profile.

This paper presents new algorithms-fuzzy c-medoids (FCMdd) and robust fuzzy c-medoids (RFCMdd)-for fuzzy clustering of relational data. The objective functions are based on selecting c representative objects (medoids) from the data set in such a way that the total fuzzy dissimilarity within each cluster is minimized. A comparison of FCMdd with the well-known relational fuzzy c-means algorithm (RFCM) shows that FCMdd is more efficient. We present several applications of these algorithms to Web mining, including Web document clustering, snippet clustering, and Web access log analysis.

We propose a new algorithm that performs clustering and feature weighting simultaneously and in an unsupervised manner. The proposed algorithm is computationally and implementationally simple, and learns a different set of feature weights for each cluster. The cluster dependent feature weights have two advantages. First, they help in partitioning the data set into more meaningful clusters. Second, they can be used as part of a more complex learning system to enhance its learning behavior. The performance of the proposed algorithm is illustrated by using it to segment real color images.

We present a new unsupervised robust clustering algorithm that can successfully find dense areas (clusters) in feature space and determine their number. The clustering problem is converted to a multimodal function optimization problem within the context of genetic niching. The niche peaks, which constitute the final cluster centers, are identified based on deterministic crowding (DC). The problem of crossover interactions in DC is eliminated by restricting the mating to members of the same niche only. Finally, the correct number of niche peaks or cluster centers is extracted from the final population. Genetic optimization makes our approach much less prone to suboptimal solutions than other objective function based approaches, and frees it from the necessity of an analytical derivation of the prototypes. As a result, our approach can handle a vast array of general subjective, even non-metric dissimilarities, and is thus useful in many applications such as Web and data mining. Additionally, the use of robust weights makes it less sensitive to the presence of noise than most traditional unsupervised clustering techniques.

Traditionally, prototype-based fuzzy clustering algorithms such as the Fuzzy C Means (FCM) algorithm have been used to find "compact" or "filled" clusters. Recently, there have been attempts to generalize such algorithms to the case of hollow or "shell-like" clusters, i.e., clusters that lie in subspaces of feature space. The shell clustering approach provides a powerful means to solve the hitherto unsolved problem of simultaneously fitting multiple curves/surfaces to unsegmented, scattered and sparse data. In this paper, we present several fuzzy and possibilistic algorithms to detect linear and quadric shell clusters. We also introduce generalizations of these algorithms in which the prototypes represent sets of higher-order polynomial functions. The suggested algorithms provide a good trade-off between computational complexity and performance, since the objective function used in these algorithms is the sum of squared distances, and the clustering is sensitive to noise and outliers. We show that by using a possibilistic approach to clustering, one can make the proposed algorithms robust.>

Shell clustering algorithms are ideally suited for computer vision tasks such as boundary detection and surface approximation, particularly when the boundaries have jagged or scattered edges and when the range data is sparse. This is because shell clustering is insensitive to local aberrations, it can be performed directly in image space, and unlike traditional approaches it does assume dense data and does not use additional features such as curvatures and surface normals. The shell clustering algorithms introduced in Part I of this paper assume that the number of clusters is known, however, which is not the case in many boundary detection and surface approximation applications. This problem can be overcome by considering cluster validity. We introduce a validity measure called surface density which is explicitly meant for the type of applications considered in this paper, we show through theoretical derivations that surface density is relatively invariant to size and partiality (incompleteness) of the clusters. We describe unsupervised clustering algorithms that use the surface density measure and other measures to determine the optimum number of shell clusters automatically, and illustrate the application of the proposed algorithms to boundary detection in the case of intensity images and to surface approximation in the case of range images.>

Evidence Aggregation Networks based on multiplicative fuzzy hybrid operators were introduced by Krishnapuram and Lee. They have been used for image segmentation, pattern recognition, and general multicriteria decision making. One of the drawbacks to these networks is that the training is complex and quite time consuming. In this article, we modify these aggregation networks to implement additive fuzzy hybrid connectives. We study the theoretical properties of two classes of such aggregation operators, one where the union and intersection components are based on multiplication, and the other where these components are derived from Yager connectives. These new networks have similar excellent properties such as backpropagation training and node interpretability for decision making under uncertainty as do their multiplicative precursors. They also have the advantage that training is easier since the derivatives of the additive hybrid operators are not as complex in form. the appropriate training algorithms are derived, and several examples given to illustrate the properties of the networks. © 1994 John Wiley & Sons, Inc.

The fuzzy c spherical shells (FCSS) algorithm is specially designed to search for clusters that can be described by circular arcs or, generally, by shells of hyperspheres. A new approach to the FCSS algorithm is presented. This algorithm is computationally and implementationally simpler than other clustering algorithms that have been suggested for this purpose. An unsupervised algorithm which automatically finds the optimum number of clusters is not known. It uses a cluster validity measure to identify good clusters, merges all compatible clusters, and eliminates spurious clusters to achieve the final results. Experimental results on several data sets are presented.