Time series anomaly detection has garnered significant research attention due to growing demands for temporal data monitoring across diverse domains. Despite the rapid advent of unsupervised anomaly detection models, existing approaches face two critical challenges in understanding the mechanisms of reconstruction-based models when handling diverse temporal dependencies: (1) the insufficient exploration of complex inter-timestamp relationships encompassing both short-term and long-term dependencies, and (2) the lack of integrated frameworks for jointly learning short-term patterns and long-term temporal characteristics. To address these challenges, we propose the novel Multi-Scale Hypergraph Transformer (MSHTrans), which leverages the capacity of hypergraphs for modeling multi-order temporal dependencies. Particularly, our method employs multi-scale downsampling to derive complementary fine-grained and coarse-grained representations, integrated with trainable hypergraph neural networks that can adaptively learn inter-timestamp relationships. The framework further integrates time series decomposition to systematically extract periodic and trend components from multi-granular features, thereby enhancing long-term dependency modeling. Through synergistic integration of learned short-term patterns and long-term temporal structures, the model achieves comprehensive time series reconstruction for effective anomaly detection. Extensive experiments demonstrate that MSHTrans outperforms state-of-the-art competitors with an average performance improvement of 8.21% (without point adjustment) and 3.52% (with point adjustment).

Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, highly precise alignment remains challenging, especially for nodes with long-range connectivity to labeled anchors. To alleviate this limitation, we propose WL-Align which employs a regularized representation learning framework to learn distinctive node representations. It extends the Weisfeiler-Lehman Isormorphism Test and learns the alignment in alternating phases of “across-network Weisfeiler-Lehman relabeling” and “proximity-preserving representation learning”. The across-network Weisfeiler-Lehman relabeling is achieved through iterating the anchor-based label propagation and a similarity-based hashing to exploit the known anchors’ connectivity to different nodes in an efficient and robust manner. The representation learning module preserves the second-order proximity within individual networks and is regularized by the across-network Weisfeiler-Lehman hash labels. Extensive experiments on real-world and synthetic datasets have demonstrated that our proposed WL-Align outperforms the state-of-the-art methods, achieving significant performance improvements in the “exact matching” scenario.

Predictive analysis of electronic health records (EHR) is a fundamental task that could provide actionable insights to help clinicians improve the efficiency and quality of care. EHR are commonly recorded in binary format and contain inevitable missing data. The nature of missingness may vary by patients, clinical features, and time, which incurs observation bias. It is essential to account for the binary missingness and observation bias or the predictive performance could be substantially compromised. In this paper, we develop a propensity-adjusted temporal network (PATNet) to conduct data imputation and predictive analysis simultaneously. PATNet contains three subnetworks: 1) an imputation subnetwork that generates the initial imputation based on historical observations, 2) a propensity subnetwork that infers the patient-, feature-, and time-dependent propensity scores, and 3) a prediction subnetwork that produces the missing-informative prediction using the propensity-adjusted imputations and the missing probabilities. To allow the propensity scores to be inferred from data, we use the expectation-maximization (EM) algorithm to learn the imputation and propensity subnetworks and incorporate a low-rank constraint via PARAFAC2 approximation. Extensive evaluation using the MIMIC-III and eICU datasets demonstrates that PATNet outperforms the state-of-the-art methods in terms of binary data imputation, disease progression modeling, and mortality prediction tasks.

Representation learning has been applied to Electronic Health Records (EHR) for medical concept embedding and the downstream predictive analytics tasks with promising results. Medical ontologies can also be integrated to guide the learning so the embedding space can better align with existing medical knowledge. Yet, properly carrying out the integration is non-trivial. Medical concepts that are similar according to a medical ontology may not be necessarily close in the embedding space learned from the EHR data, as medical ontologies organize medical concepts for their own specific objectives. Any integration methodology without considering the underlying inconsistency will result in sub-optimal medical concept embedding and, in turn, degrade the performance of the downstream tasks. In this article, we propose a novel representation learning framework called ADORE ( AD aptive O ntological RE presentations) that allows the medical ontologies to adapt their structures for more robust integrating with the EHR data. ADORE first learns multiple embeddings for each category in the ontology via an attention mechanism. At the same time, it supports an adaptive integration of categorical and multi-relational ontologies in the embedding space using a category-aware graph attention network. We evaluate the performance of ADORE on a number of predictive analytics tasks using two EHR datasets. Our experimental results show that the medical concept embeddings obtained by ADORE can outperform the state-of-the-art methods for all the tasks. More importantly, it can result in clinically meaningful sub-categorization of the existing ontological categories and yield attention values that can further enhance the model interpretability.

Social network alignment aims at aligning person identities across social networks. Embedding based models have been shown effective for the alignment where the structural proximity preserving objective is typically adopted for the model training. With the observation that “overly-close” user embeddings are unavoidable for such models causing alignment inaccuracy, we propose a novel learning framework which tries to enforce the resulting embeddings to be more widely apart among the users via the introduction of carefully implanted pseudo anchors. We further proposed a meta-learning algorithm to guide the updating of the pseudo anchor embeddings during the learning process. The proposed intervention via the use of pseudo anchors and meta-learning allows the learning framework to be applicable to a wide spectrum of network alignment methods. We have incorporated the proposed learning framework into several state-of-the-art models. Our experimental results demonstrate its efficacy where the methods with the pseudo anchors implanted can outperform their counterparts without pseudo anchors by a fairly large margin, especially when there only exist very few labeled anchors.

Data summarization that presents a small subset of a dataset to users has been widely applied in numerous applications and systems. Many datasets are coded with hierarchical terminologies, e.g., gene ontology, disease ontology, to name a few. In this paper, we study the weighted tree summarization. We motivate and formulate our${\mathsf {kWTS}}$-${\mathsf {problem}}$as selecting a diverse set of$k$nodes tosummarize a hierarchicaltree$T$withweighted terminologies. We first propose an efficient greedy tree summarization algorithm${\mathsf {GTS}}$. It solves the problem with$(1-1/e)$-approximation guarantee. Although${\mathsf {GTS}}$achieves quality-guaranteed answers approximately, but it is still not optimal. To tackle the problem optimally, we further develop a dynamic programming algorithm${\mathsf {OTS}}$to obtain optimal answers for${\mathsf {kWTS}}$-${\mathsf {problem}}$in$O(nhk^3)$time, where$n, h$are the node size and height in tree$T$. The algorithm complexity and correctness of${\mathsf {OTS}}$are theoretically analyzed. In addition, we propose a useful optimization technique of tree reduction to remove useless nodes with zero weights and shrink the tree into a smaller one, which ensures the efficiency acceleration of both${\mathsf {GTS}}$and${\mathsf {OTS}}$in real-world datasets. Moreover, we illustrate one useful application of graph visualization based on the answer of$k$-sized tree summarization and show it in a novel case study. Extensive experimental results on real-world datasets show the effectiveness and efficiency of our proposed approximate and optimal algorithms for tree summarization. Furthermore, we conduct a usability evaluation of attractive topic recommendation on ACM Computing Classification System dataset to validate the usefulness of our model and algorithms.

Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenotyping models.

PARAFAC2 factorization provides a practical solution to map the temporally irregular electronic health records (EHR) to clinically relevant and interpretable phenotypes.Existing methods ignore the effect of interdependency of diseases over clinical history.Consequently, the crucial temporal information contained in the EHR data cannot be fully utilized and the learned phenotypes can be sub-optimal to characterize patients with progressive conditions.To address this issue, we propose a novel temporally dependent PARAFAC2 (TedPar) factorization in which the temporal dependency among the phenotypes is explicitly modeled.TedPar learns a set of target phenotypes to capture the clinical features relevant to the diseases of interest and a set of background phenotypes to capture irrelevant but frequently co-occurring clinical features.By effectively modeling the temporal dependency and separating relevant and irrelevant features, the discovered target phenotypes can be used to model the progression of the diseases of interest.Empirical evaluations show that TedPar obtains up to 32.4% relative improvement in reconstruction accuracy over the test set, suggesting significantly better generalizability than the baselines for both noise-free and heavily noisy input data.Qualitative analysis also shows that TedPar is capable of discovering clinically meaningful phenotypes and capturing the temporal dependency between them.

Properly analyzing health insurance claims data could lead to significant business insights and benefits for health service providers and insurance companies. Yet, health insurance data is often high dimensional and contains complex interleave sequences of claims. Instead of conducting machine learning tasks directly on the raw data, a better approach is performing the tasks on high-quality embeddings of the raw data. Driven by the real business need of Solution Segic Inc., a Canadian technology company in the group insurance industry, we extract health insurance claims embeddings with neural networks in the context of anomaly detection. We propose and thoroughly examine six embedding components that are customized based on different possible assumptions made on the data. One of our proposed embedding components, EC-ReStepRec, significantly outperforms other candidates on two anomaly detection tasks. This is the first embedding study done on health insurance claims for anomaly detection.

Binary data with one-class missing values are ubiquitous in real-world applications. They can be represented by irregular tensors with varying sizes in one dimension, where value one means presence of a feature while zero means unknown (i.e., either presence or absence of a feature). Learning accurate low-rank approximations from such binary irregular tensors is a challenging task. However, none of the existing models developed for factorizing irregular tensors take the missing values into account, and they assume Gaussian distributions, resulting in a distribution mismatch when applied to binary data. In this paper, we propose Logistic PARAFAC2 (LogPar) by modeling the binary irregular tensor with Bernoulli distribution parameterized by an underlying real-valued tensor. Then we approximate the underlying tensor with a positive-unlabeled learning loss function to account for the missing values. We also incorporate uniqueness and temporal smoothness regularization to enhance the interpretability. Extensive experiments using large-scale real-world datasets show that LogPar outperforms all baselines in both irregular tensor completion and downstream predictive tasks. For the irregular tensor completion, LogPar achieves up to 26% relative improvement compared to the best baseline. Besides, LogPar obtains relative improvement of 13.2% for heart failure prediction and 14% for mortality prediction on average compared to the state-of-the-art PARAFAC2 models.

Aligning users across different social networks has become increasingly studied as an important task to social network analysis. In this paper, we propose a novel representation learning method that mainly exploits social structures for the network alignment. In particular, the proposed network embedding framework models the follower-ship and followee-ship of each user explicitly as input and output context vectors, while preserving the proximity of users with “similar” followers and followees in the embedded space. We incorporate both known and predicted user anchors across the networks as constraints to facilitate the transfer of context information to achieve accurate user alignment. Both network embedding and user alignment are inferred under a unified optimization framework with negative sampling adopted to ensure scalability. Also, variants of the proposed framework, including the incorporation of higher-order structural features, are also explored for further boosting the alignment accuracy. Extensive experiments on large-scale social and academia network datasets demonstrate the efficacy of our proposed model compared with state-of-the-art methods.

Scientific workflows define computational processes needed for carrying out scientific experiments. Existing workflow repositories contain hundreds of scientific workflows, where scientists can find materials and knowledge to facilitate workflow design for running related experiments. Identifying reusable fragments in growing workflow repositories has become increasingly important. In this paper, we present PSM-Flow, a probabilistic subgraph mining algorithm designed to discover commonly occurring fragments in a workflow corpus using a modified version of the Latent Dirichlet Allocation algorithm. The proposed model encodes the geodesic distance between workflow steps into the model for implicitly modeling fragments. PSM-Flow captures variations of frequent fragments while maintaining its space complexity bounded polynomially, as it requires no candidate generation. We applied PSM-Flow to three real-world scientific workflow datasets containing more than 750 workflows for neuroimaging analysis. Our results show that PSM-Flow outperforms three state of the art frequent subgraph mining techniques. We also discuss other potential future improvements of the proposed method.

The elderly living in smart homes can have their daily movement recorded and analyzed. As different elders can have their own living habits, a methodology that can automatically identify their daily activities and discover their daily routines will be useful for better elderly care and support. In this article, we focus on automatic detection of behavioral patterns from the trajectory data of an individual for activity identification as well as daily routine discovery. The underlying challenges lie in the need to consider longer-range dependency of the sensor triggering events and spatiotemporal variations of the behavioral patterns exhibited by humans. We propose to represent the trajectory data using a behavior-aware flow graph that is a probabilistic finite state automaton with its nodes and edges attributed with some local behavior-aware features. We identify the underlying subflows as the behavioral patterns using the kernel k -means algorithm. Given the identified activities, we propose a novel nominal matrix factorization method under a Bayesian framework with Lasso to extract highly interpretable daily routines. For empirical evaluation, the proposed methodology has been compared with a number of existing methods based on both synthetic and publicly available real smart home datasets with promising results obtained. We also discuss how the proposed unsupervised methodology can be used to support exploratory behavior analysis for elderly care.

Data summarization that presents a small subset of a dataset to users has been widely applied in numerous applications and systems. Many datasets are coded with hierarchical terminologies, e.g., the international classification of Diseases-9, Medical Subject Heading, and Gene Ontology, to name a few. In this paper, we study the problem of selecting a diverse set of k elements to summarize an input dataset with hierarchical terminologies, and visualize the summary in an ontology structure. We propose an efficient greedy algorithm to solve the problem with (1-1/e)≈ 62%-approximation guarantee. Preliminary experimental results on real-world datasets show the effectiveness and efficiency of the proposed algorithm for data summarization.

With the advent of the Internet of things (IoT) and smart sensor technologies, the data-driven paradigm has been found promising to support human behavioral analysis in a smart home for better healthcare and well-being of senior adults. This work focuses on discovering daily activity routines from sensor data collected in a smart home. By representing the sensor data as a matrix, daily activity routines can be identified using matrix factorization methods. The key challenge rests on the fact that the matrix contains discrete labels as its elements, and decomposing the nominal data matrix into basis vectors of the labels is nontrivial. We propose a novel principled methodology to tackle the nominal matrix factorization problem. Assuming that the similarity matrix of the labels is known, the discrete labels are first projected onto a continuous space with the interlabel distance preserving the given similarity matrix of the labels as far as possible. Then, we extend a hierarchical probabilistic model for ordinal matrix factorization with Bayesian Lasso that the factorization can be more robust to noise and more sparse to ease human interpretation. Our experimental results based on a synthetic data set shows that the factorization results obtained using the proposed methodology outperform those obtained using a number of the state-of-the-art factorization methods in terms of the basis vector reconstruction accuracy. We also applied our model to a publicly available smart home data set to illustrate how the proposed methodology can be used to support daily activity routine analysis.

Location-based social networks have been increasingly used to experience users new possibilities, including personalized point-of-interest (POI) recommendation services which leverages on the overlapping of user trajectories to recommend POI collaboratively. POI recommendation is challenging as it does not just suffers from the problems known for collaborative filtering such as data sparsity and cold-start, but to a much greater extent. Most of the related works apply the conventional recommendation approaches to POI recommendation while overlooking the personalized time-variant human behavioral tendency. In this paper, we put forward a tensor factorization-based ranking methodology to recommend users their interested locations by considering their time-varying behavioral trends. We also propose to categorize the locations to address data sparsity and cold-start issues, and accordingly new locations the user have not been visited can thus be bubbled up during ranking the location candidates. The tensor factorization is carefully studied to prune the irrelevant factors to the ranking results to achieve efficient POI recommendation. The experimental results validate the effectiveness of our proposed mechanism which outperforms the state-of-the-art approaches by over 8% for precision.

Epidemic prediction is an important problem in epidemic control. Poisson regression methods are often adopted in existing works, mostly with only the (intra-)regional environmental factors considered. As the diffusion of epidemics is affected by not only the intra-regional factors but also inter-regional and external ones, a unified framework based on Poisson regression with the three types of factors incorporated is proposed for the prediction. Specifically, we propose a Poisson-regression-based model first with the intra-regional and inter-regional factors included. The intra-regional factor in a particular time interval is represented by one feature vector with the regionally environmental and social factors considered. The inter-regional factor is modeled by a diffusion matrix which describes the possibilities that the epidemics can spread from one region to another, which in turn accounts for the propagating effects of the infected cases. To learn the structure of the diffusion matrix, we propose two approaches-utilizing some a priori knowledge (e.g., transportation network) and estimating it from scratch via a sparse structure assumption. The resulting optimization problem of the maximum a posterior solution is a convex one and can be efficiently solved by the alternating direction method of multipliers (ADMM). In addition, we incorporate also the external factor, i.e., the imported cases. With one fact that the distribution of the number of infected cases over a year is (approximately) unimodal for most epidemics and one assumption that the importing rate has a small variance over the year, we can approximate the effect of the external factor with a parametric function (e.g., a quadratic function) over time. The resulting optimization problem is still convex and can be also solved by the ADMM algorithm. Empirical evaluations are conducted based on a real data set which records the 16-days-reported cases in the Yunnan province of China for seven years, from 2005 to 2011. The experimental results demonstrate the effectiveness of our proposed models.

Many real-world problems exhibit phenomena which are best represented as complex networks with dynamic structures (e.g., human communication networks). Network motifs have been shown effective for characterizing the structural properties of such complex networks. Nevertheless, related motif models typically do not consider stochastic structural and sequential variations, hinting their limitations on dynamic network analysis. In this paper, we consider networks with time-stamped edges and model their local structural and temporal variations using a mixture of Markov chains for stochastic temporal network motif detection. The optimal number of motifs is automatically estimated in a Bayesian framework. We evaluated the proposed method using synthetic networks and found to be robust against noise compared to the deterministic approach. Also, we applied it to a mobile phone usage data set to demonstrate how the human communication patterns embedded in the data set can be detected. In addition, we make use of a hidden Markov model with different distributions for the mixing proportions of the motifs defining its states, and demonstrated how the evolution of the communication patterns can also be identified.

Diffusion is known to be an important process governing the behaviours observed in network environments like social networks, contact networks, etc. For modeling the diffusion process, the Independent Cascade Model (IC Model) is commonly adopted and algorithms have been proposed for recovering the hidden diffusion network based on observed cascades. However, the IC Model assumes the effects of multiple neighbors on a node to be independent and does not consider the structural diversity of nodes' neighbourhood. In this paper, we propose an extension of the IC Model with the community structure of node neighbours incorporated. We derive an expectation maximization (EM) algorithm to infer the model parameters. To evaluate the effectiveness and efficiency of the proposed method, we compared it with the IC model and its variants that do not consider the structural properties. Our empirical results based on the MemeTracker dataset, shows that after incorporating the structural diversity, there is a significant improvement in the modelling accuracy, with reasonable increase in run-time.

This paper presents the coauthor network topic (CNT) model constructed based on Markov random fields (MRFs) with higher-order cliques. Regularized by the complex coauthor network structures, the CNT can simultaneously learn topic distributions as well as expertise of authors from large document collections. Besides modeling the pairwise relations, we model also higher-order coauthor relations and investigate their effects on topic and expertise modeling. We derive efficient inference and learning algorithms from the Gibbs sampling procedure. To confirm the effectiveness, we apply the CNT to the expert finding problem on a DBLP corpus of titles from six different computer science conferences. Experiments show that the higher-order relations among coauthors can improve the topic and expertise modeling performance over the case with pairwise relations, and thus can find more relevant experts given a query topic or document.

In this paper we propose the multirelational topic model (MRTM) for multiple types of link modeling such as citation and coauthor links in document networks. In the citation network, the MRTM models the citation link between each pair of documents as a binary variable conditioned on their topic distributions. In the coauthor network, the MRTM models the coauthor link between each pair of authors as a binary variable conditioned on their expertise distributions. The topic discovery is collectively regularized by multiple relations in both citation and coauthor networks. This model can summarize topics from the document network, predict citation links between documents and coauthor links between authors. Efficient inference and learning algorithms are derived based on Gibbs sampling. Experiments demonstrate that the MRTM significantly outperforms other state-of-the-art single-relational link modeling methods for large scientific document networks.

Real-world social networks (e.g., blogosphere) often evolve over time and thus poses challenges on conventional social network analysis techniques which model the underlying networks as static graphs. In this paper, we are interested in detecting dynamic communities and their trend of evolution in a social network by examining the structural and dynamic patterns of interactions. In doing so, we propose an iterative mining algorithm for computing the intensities and bursts of some hidden communities over time. Our method is probabilistic in nature and can be applied to both undirected graphs and directed graphs. Quantitative and qualitative performance comparisons between the proposed method and some representative methods for social network analysis are provided. Evaluation results based on three benchmark datasets, including Reuters terror news network, political blogosphere, and Enron emails, show that the proposed method is both effective and efficient.

Many complex systems in the real world can be modeled as signed social networks that contain both positive and negative relations. Algorithms for mining social networks have been developed in the past, however most of them were designed primarily for networks containing only positive relations and thus not suitable for signed networks. In this work, we propose a new algorithm, called FEC, to mine signed social networks so that both positive within-group relations and negative between-group relations are dense. FEC considers both the sign and the density of relations as the clustering attributes, making itself effective for not only signed networks but also conventional social networks including only positive relations. Also, FEC adopts an agent-based heuristic that makes the algorithm efficient (in linear time with respect to the size of a network) and capable of giving nearly optimal solutions. FEC depends on only one parameter whose value can easily be set, and requires no prior knowledge on hidden community structures. The effectiveness and efficacy of FEC have been demonstrated through a set of rigorous experiments involving both benchmark and randomly-generated signed networks.

Mining distributed data for global knowledge is getting more attention recently. The problem is especially challenging when data sharing is prohibited due to local constraints like limited bandwidth and data privacy. In this paper, we investigate how to derive the embedded manifold (as a 2-D map) for a horizontally partitioned data set, where data cannot be shared among the partitions directly. We propose a model-based approach which computes hierarchical local data abstractions, aggregates the abstractions, and finally learns a global generative model - generative topographic mapping (GTM) based on the aggregated data abstraction. We applied the proposed method to two benchmarking data sets and demonstrated that the accuracy of the derived manifold can effectively be controlled by adjusting the data granularity level of the adopted local abstraction.

Structured link vector model (SLVM) is a recently proposed document representation that takes into account both structural and semantic information for measuring XML document similarity. Its formulation includes an element similarity matrix for capturing the semantic similarity between XML elements - the structural components of XML documents. In this paper, instead of applying heuristics to define the similarity matrix, we proposed to learn the matrix using pair wise similar training data in an iterative manner. In addition, we extended SLVM to SLVM-LSI by incorporating term semantics into SLVM using latent semantic indexing, with the element similarity related properties of the original SLVM preserved. For performance evaluation, we applied SLVM-LSI to similarity-based clustering of two XML datasets and the proposed SLVM-LSI was found to significantly outperform the conventional vector space model and the edit-distance based methods. The similarity matrix, obtained as a byproduct via the learning, can provide higher level knowledge about the semantic relationship between the XML elements.

While numerous efforts have focused on service composition in the Grid environment, service selection among similar services from multiple providers has not been addressed. In particular, all service composition work done so far are based on a given selection of services under a well set environment. As a result, uncertainty (e.g., server load, network traffic, computation time of the services due to changing memory and other unexpected conditions) under a real, dynamic environment has never been considered. This paper prototypes the service selection under a Grid environment and proposes an uncertainty framework to address the issue. Experimental results show that our considerations are valid and our preliminary solution works well in our Globus Grid network.

Distributed data mining has been a topic getting more important nowadays as there are many cases where physically sharing of data is probibited, e.g., due to huge data volume or data privacy. In this paper, we are interested in learning a global cluster model by exploring data in distributed sources. A methodology based on periodic model exchange and merge is proposed and applied to hyperlinked Web pages analysis. In addition, we have tested a number of variations of the basic idea, including putting more emphasis on the privacy concern and testing the effect of having different numbers of distributed sources. Experimental results show that the proposed distributed learning scheme is effective with accuracy close to the case with all the data physically shared for the learning.

The Centre for e-Transformation Research (CTR), also an affiliated Centre of Web Intelligence Consortium (WIC), is established under the Science Faculty of Hong Kong Baptist University, currently funded by Hong Kong Research Grant Council Central Allocation and FRG Strategic Research Grant, Hong Kong Baptist University for developing an Area of Strength in e-transformation research, making high impact to various sectors of the society, from e-business, e-learning, to e-government, to name a few.

Foraging information in large and complex Web sites simply using keyword search usually results in unpleasant experience due to the overloaded search results. To support more effective information search, some descriptive abstractions of the Web sites (e.g., sitemaps) are mostly needed. However, their creation and maintenance normally requires recurrent manual effort due to the fast-changing Web contents. We extend the HITS algorithm and integrate hyperlink topology and Web site hierarchy to identify a hierarchy of Web page clusters as the abstraction of a Web site. As the algorithm is based on HITS, each identified cluster follows the bipartite graph structure, with an authority and hub pair as the cluster summary. The effectiveness of the algorithm has been evaluated using three different Web sites (containing /spl sim/6000-14000 Web pages) with promising results. Detailed interpretation of the experimental results as well as qualitative comparison with other related works are also included.