OBJECTIVES: To investigate approaches of reasoning with large language models (LLMs) and to propose a new prompting approach, ensemble reasoning, to improve medical question answering performance with refined reasoning and reduced inconsistency. MATERIALS AND METHODS: We used multiple choice questions from the USMLE Sample Exam question files on 2 closed-source commercial and 1 open-source clinical LLM to evaluate our proposed approach ensemble reasoning. RESULTS: On GPT-3.5 turbo and Med42-70B, our proposed ensemble reasoning approach outperformed zero-shot chain-of-thought with self-consistency on Steps 1, 2, and 3 questions (+3.44%, +4.00%, and +2.54%) and (2.3%, 5.00%, and 4.15%), respectively. With GPT-4 turbo, there were mixed results with ensemble reasoning again outperforming zero-shot chain-of-thought with self-consistency on Step 1 questions (+1.15%). In all cases, the results demonstrated improved consistency of responses with our approach. A qualitative analysis of the reasoning from the model demonstrated that the ensemble reasoning approach produces correct and helpful reasoning. CONCLUSION: The proposed iterative ensemble reasoning has the potential to improve the performance of LLMs in medical question answering tasks, particularly with the less powerful LLMs like GPT-3.5 turbo and Med42-70B, which may suggest that this is a promising approach for LLMs with lower capabilities. Additionally, the findings show that our approach helps to refine the reasoning generated by the LLM and thereby improve consistency even with the more powerful GPT-4 turbo. We also identify the potential and need for human-artificial intelligence teaming to improve the reasoning beyond the limits of the model.

Networks are used as highly expressive tools in different disciplines. In recent years, the analysis and mining of temporal networks have attracted substantial attention. Frequent pattern mining is considered an essential task in the network science literature. In addition to the numerous applications, the investigation of frequent pattern mining in networks directly impacts other analytical approaches, such as clustering, quasi-clique and clique mining, and link prediction. In nearly all the algorithms proposed for frequent pattern mining in temporal networks, the networks are represented as sequences of static networks. Then, the inter- or intra-network patterns are mined. This type of representation imposes a computation-expressiveness trade-off to the mining problem. In this paper, we propose a novel representation that can preserve the temporal aspects of the network losslessly. Then, we introduce the concept of constrained interval graphs ( CIGs). Next, we develop a series of algorithms for mining the complete set of frequent temporal patterns in a temporal network data set. We also consider four different definitions of isomorphism for accommodating minor variations in temporal data of networks. Implementing the algorithm for three real-world data sets proves the practicality of the proposed approach and its capability to discover unknown patterns in various settings.

Networks are known as perfect tools for modeling various types of systems. In the literature of network mining, frequent subgraph mining is considered as the essence of mining network data. In this problem, the dataset is composed of networks representing multiple independent systems or one system at multiple time stamps. The cores of mining frequent subgraphs are graph and subgraph isomorphism. Due to the complexities of these problems, the frequent subgraph mining algorithms proposed in the literature employ various heuristics for candidate generation, duplicate subgraphs pruning, and support computation. In this survey, we provide a classification of proposed algorithms in the literature. The algorithms for static networks have found numerous applications. Therefore, these algorithms will be reviewed in detail. Besides, it is discussed that consideration of temporality of data can impact the derived insight and attracted substantial attention in recent years. However, prior surveys have not comprehensively examined the algorithms of frequent subgraph mining in a database of temporal networks represented as network snapshots. Therefore, the algorithms proposed for mining frequent subgraphs in temporal networks are reviewed. Moreover, most of the surveys have focused on main-memory algorithms. Here, we review disk-based, parallel, and distributed algorithms proposed for mining frequent subgraphs.

Sepsis is a devastating multi-stage health condition with a high mortality rate. Its complexity, prevalence, and dependency of its outcomes on early detection have attracted substantial attention from data science and machine learning communities. Previous studies rely on individual cellular and physiological responses representing organ system failures to predict health outcomes or the onset of different sepsis stages. However, it is known that organ systems' failures and dynamics are not independent events. In this study, we identify the dependency patterns of significant proximate sepsis-related failures of cellular and physiological responses using data from 12,223 adult patients hospitalized between July 2013 and December 2015. The results show that proximate failures of cellular and physiological responses create better feature sets for outcome prediction than individual responses. Our findings reveal the few significant proximate failures that play the major roles in predicting patients' outcomes. This study's results can be simply translated into clinical practices and inform the prediction and improvement of patients' conditions and outcomes.

Online health communities (OHCs) provide health consumers with platforms for discussing medical conditions and sharing a personal experience. Although a wealth of healthcare information is available in OHCs, consumers find it challenging to locate information of interest efficiently due to the information overload. The lack of medical knowledge and searching skills makes it even harder for consumers to retrieve demanded information from a popular OHC with hundreds of thousands of threads. Therefore, effective thread recommendation is critical for OHCs to enhance user experience and engage the users in the community. In this paper, we proposed to recommend threads to users in OHCs by exploiting heterogeneous healthcare information network mining. We first constructed a heterogeneous healthcare information network from OHCs data. Unlike bipartite graphs studied in most existing works, which only consider user nodes and item nodes, a heterogeneous healthcare information network retains the rich context information of users and threads. We extracted features from the network to capture basic network metrics, thread-thread relationship, and user-user relationship, and utilize the features to train a binary classification model for thread recommendation. Experiments were conducted using a data set collected from MedHelp. The proposed approach was proven to be effective in measuring user interests in online discussion threads. In addition, by testing our approaches using different settings, we found that the local similarity achieved better performance than the global similarity in heterogeneous information network. By incorporating thread- thread relationship and user-user relationship, it can achieve the best performance.

Most cancer patients, including patients with breast cancer, experience multiple symptoms simultaneously while receiving active treatment. Some symptoms tend to occur together and may be related, such as hot flashes and night sweats. Co-occurring symptoms may have a multiplicative effect on patients' functioning, mental health, and quality of life. Symptom clusters in the context of oncology were originally described as groups of three or more related symptoms. Some authors have suggested symptom clusters may have practical applications, such as the formulation of more effective therapeutic interventions that address the combined effects of symptoms rather than treating each symptom separately. Most studies that have sought to identify clusters in breast cancer survivors have relied on traditional research studies. Social media, such as online health-related forums, contain a bevy of user-generated content in the form of threads and posts, and could be used as a data source to identify and characterize symptom clusters among cancer patients. The present study seeks to determine patterns of symptom clusters in breast cancer survivors derived from both social media and research study data using improved K-Medoid clustering. A total of 50,426 publicly available messages were collected from Medhelp.com and 653 questionnaires were collected as part of a research study. The network of symptoms built from social media was sparse compared to that of the research study data, making the social media data easier to partition. The proposed revised K-Medoid clustering helps to improve the clustering performance by re-assigning some of the negative-ASW (average silhouette width) symptoms to other clusters after initial K-Medoid clustering. This retains an overall non-decreasing ASW and avoids the problem of trapping in local optima. The overall ASW, individual ASW, and improved interpretation of the final clustering solution suggest improvement. The clustering results suggest that some symptom clusters are consistent across social media data and clinical data, such as gastrointestinal (GI) related symptoms, menopausal symptoms, mood-change symptoms, cognitive impairment and pain-related symptoms. We recommend an integrative approach taking advantage of both data sources. Social media data could provide context for the interpretation of clustering results derived from research study data, while research study data could compensate for the risk of lower precision and recall found using social media data.

Informational support and nurturant support are two basic types of social support offered in online health communities. This study identifies types of social support in the QuitStop forum and brings insights to exchange patterns of social support and user behaviors with content analysis and social network analysis. Motivated by user information behavior, this study defines two patterns to describe social support exchange: initiated support exchange and invited support exchange. It is found that users with a longer quitting time tend to actively give initiated support, and recent quitters with a shorter abstinent time are likely to seek and receive invited support. This study also finds that support givers of informational support quit longer ago than support givers of nurturant support, and support receivers of informational support quit more recently than support receivers of nurturant support. Usually, informational support is offered by users at late quit stages to users at early quit stages. Nurturant support is also exchanged among users within the same quit stage. These findings help us understand how health consumers are supporting each other and reveal new capabilities of online intervention programs that can be designed to offer social support in a timely and effective manner.

Mining features and opinion words is essential for fine-grained opinion analysis of customer reviews. It is observed that semantic dependencies naturally exist between features and opinion words, even among features or opinion words themselves. In this article, we employ a corpus statistics association measure to quantify the pairwise word dependencies and propose a generalized association-based unified framework to identify features, including explicit and implicit features, and opinion words from reviews. We first extract explicit features and opinion words via an association-based bootstrapping method (ABOOT). ABOOT starts with a small list of annotated feature seeds and then iteratively recognizes a large number of domain-specific features and opinion words by discovering the corpus statistics association between each pair of words on a given review domain. Two instances of this ABOOT method are evaluated based on two particular association models, likelihood ratio tests (LRTs) and latent semantic analysis (LSA). Next, we introduce a natural extension to identify implicit features by employing the recognized known semantic correlations between features and opinion words. Experimental results illustrate the benefits of the proposed association-based methods for identifying features and opinion words versus benchmark methods.

Since adverse drug reactions (ADRs) represent a significant health problem all over the world, ADR detection has become an important research topic in drug safety surveillance. As many potential ADRs cannot be detected though premarketing review, drug safety currently depends heavily on postmarketing surveillance. Particularly, current postmarketing surveillance in the United States primarily relies on the FDA Adverse Event Reporting System (FAERS). However, the effectiveness of such spontaneous reporting systems for ADR detection is not as good as expected because of the extremely high underreporting ratio of ADRs. Moreover, it often takes the FDA years to complete the whole process of collecting reports, investigating cases, and releasing alerts. Given the prosperity of social media, many online health communities are publicly available for health consumers to share and discuss any healthcare experience such as ADRs they are suffering. Such health-consumer-contributed content is timely and informative, but this data source still remains untapped for postmarketing drug safety surveillance. In this study, we propose to use (1) association mining to identify the relations between a drug and an ADR and (2) temporal analysis to detect drug safety signals at the early stage. We collect data from MedHelp and use the FDA's alerts and information of drug labeling revision as the gold standard to evaluate the effectiveness of our approach. The experiment results show that health-related social media is a promising source for ADR detection, and our proposed techniques are effective to identify early ADR signals.

E‐patients seeking information online often seek specific advice related to coping with their health condition(s) among social networking sites. They may be looking for social connectivity with compassionate strangers who may have experienced similar situations to share opinions and experiences rather than for authoritative medical information. Previous studies document distinct technological features and different levels of social support interaction patterns. It is expected that the design of the social media functions will have an impact on the user behavior of social support exchange. In this part of a multipart study, we investigate the social support types, in particular information support types, across multiple computer‐mediated communication formats (forum, journal, and notes) within an alcoholism community using descriptive content analysis on 3 months of data from a MedHelp online peer support community. We present the results of identified informational support types including advice, referral, fact, personal experiences, and opinions, either offered or requested. Fact type was exchanged most often among the messages; however, there were some different patterns between notes and journal posts. Notes were used for maintaining relationships rather than as a main source for seeking information. Notes were similar to comments made to journal posts, which may indicate the friendship between journal readers and the author. These findings suggest that users may have initially joined the MedHelp Alcoholism Community for information‐seeking purposes but continue participation even after they have completed with information gathering because of the relationships they formed with community members through social media features.

An organization performing environmental scanning generally monitors or tracks various events concerning its external environment. One of the major resources for environmental scanning is online news documents, which are readily accessible on news websites or infomediaries. However, the proliferation of the World Wide Web, which increases information sources and improves information circulation, has vastly expanded the amount of information to be scanned. Thus, it is essential to develop an effective event episode discovery mechanism to organize news documents pertaining to an event of interest. In this study, we propose two new metrics, Term Frequency × Inverse Document FrequencyTempo (TF×IDFTempo) and TF×Enhanced‐IDFTempo, and develop a temporal‐based event episode discovery (TEED) technique that uses the proposed metrics for feature selection and document representation. Using a traditional TF×IDF‐based hierarchical agglomerative clustering technique as a performance benchmark, our empirical evaluation reveals that the proposed TEED technique outperforms its benchmark, as measured by cluster recall and cluster precision. In addition, the use of TF×Enhanced‐IDFTempo significantly improves the effectiveness of event episode discovery when compared with the use of TF×IDFTempo.

Detecting evolving hidden communities within dynamic social networks has attracted significant attention recently due to its broad applications in e-commerce, online social media, security intelligence, public health, and other areas. Many community network detection techniques employ a two-stage approach to identify and detect evolutionary relationships between communities of two adjacent time epochs. These techniques often identify communities with high temporal variation, since the two-stage approach detects communities of each epoch independently without considering the continuity of communities across two time epochs. Other techniques require identification of a predefined number of hidden communities which is not realistic in many applications. To overcome these limitations, we propose the Dynamic Stochastic Blockmodel with Temporal Dirichlet Process, which enables the detection of hidden communities and tracks their evolution simultaneously from a network stream. The number of hidden communities is automatically determined by a temporal Dirichlet process without human intervention. We tested our proposed technique on three different testbeds with results identifying a high performance level when compared to the baseline algorithm.

The vast majority of existing approaches to opinion feature extraction rely on mining patterns only from a single review corpus, ignoring the nontrivial disparities in word distributional characteristics of opinion features across different corpora. In this paper, we propose a novel method to identify opinion features from online reviews by exploiting the difference in opinion feature statistics across two corpora, one domain-specific corpus (i.e., the given review corpus) and one domain-independent corpus (i.e., the contrasting corpus). We capture this disparity via a measure called domain relevance (DR), which characterizes the relevance of a term to a text collection. We first extract a list of candidate opinion features from the domain review corpus by defining a set of syntactic dependence rules. For each extracted candidate feature, we then estimate its intrinsic-domain relevance (IDR) and extrinsic-domain relevance (EDR) scores on the domain-dependent and domain-independent corpora, respectively. Candidate features that are less generic (EDR score less than a threshold) and more domain-specific (IDR score greater than another threshold) are then confirmed as opinion features. We call this interval thresholding approach the intrinsic and extrinsic domain relevance (IEDR) criterion. Experimental results on two real-world review domains show the proposed IEDR approach to outperform several other well-established methods in identifying opinion features.

Interdisciplinary research has been attracting more attention in recent decades. In this article, we compare the similarity between scientific research domains and quantifying the temporal similarities of domains. We narrowed our study to three research domains: information retrieval (IR), database (DB), and World Wide Web (W3), because the rapid development of the W3 domain substantially attracted research efforts from both IR and DB domains and introduced new research questions to these two areas. Most existing approaches either employed a content‐based technique or a cocitation or coauthorship network‐based technique to study the development trend of a research area. In this work, we proposed an effective way to quantify the similarities among different research domains by incorporating content similarity and coauthorship network similarity. Experimental results on DBLP (DataBase systems and Logic Programming) data related to IR, DB, and W3 domains showed that the W3 domain was getting closer to both IR and DB whereas the distance between IR and DB remained relatively constant. In addition, comparing to IR and W3 with the DB domain, the DB domain was more conservative and evolved relatively slower.

The Internet is an ideal platform for business-to-consumer (B2C) and business-to-business (B2B) electronic commerce where businesses and consumers conduct commerce activities such as searching for consumer products, promoting business, managing supply chain and making electronic transactions. With the advance of Web 2.0 technologies and the popularity of social media sites, social commerce offers new opportunities of social interaction between electronic commerce consumers as well as social interaction between consumers and e-retailers. The user contributed content provides a tremendous amount of information that may assist in electronic commerce services. Social network analysis and mining has been a powerful tool for electronic commerce vendors and marketing companies to understand the user behavior which is useful for identifying potential customers of their products. However, the capability of social network analysis and mining diminishes when the social network data is incomplete, especially when there are only limited ties available. The social networks extracted from explicit relationships in social media are usually sparse. Many social media users who have similar interest may not have direct interactions with one another or purchase the same products. Therefore, the explicit relationships between electronic commerce users are not sufficient to construct social networks for effective social network analysis and mining. In this work, we propose the temporal analysis techniques to identify implicit relationships for enriching the social network structure. We have conducted an experiment on Digg.com, which is a social media site for users to discover and share content from anywhere of the Web. The experiment shows that the temporal analysis techniques outperform the baseline techniques that only rely on explicit relationships.

The rapid development of online social media sites is accompanied by the generation of tremendous web contents. Web users are shifting from data consumers to data producers. As a result, topic detection and tracking without taking users' interests into account is not enough. This paper presents a statistical model that can detect interpretable trends and topics from document streams, where each trend (short for trending story) corresponds to a series of continuing events or a storyline. A topic is represented by a cluster of words frequently co-occurred. A trend can contain multiple topics and a topic can be shared by different trends. In addition, by leveraging a Recurrent Chinese Restaurant Process (RCRP), the number of trends in our model can be determined automatically without human intervention, so that our model can better generalize to unseen data. Furthermore, our proposed model incorporates user interest to fully simulate the generation process of web contents, which offers the opportunity for personalized recommendation in online social media. Experiments on three different datasets indicated that our proposed model can capture meaningful topics and trends, monitor rise and fall of detected trends, outperform baseline approach in terms of perplexity on held-out dataset, and improve the result of user participation prediction by leveraging users' interests to different trends.

The Smart Health and Wellbeing workshop is organized to develop a platform for authors to discuss fundamental principles, algorithms or applications of intelligent data acquisition, processing and analysis of healthcare data. We are particularly interested in information and knowledge management papers, in which the approaches are accompanied by an in-depth experimental evaluation with real world data. This paper provides an overview of the workshop and the accepted contributions.

Recommender system provides users with personalized suggestions of product or information. Typically, recommender systems rely on a bipartite graph model to capture user interest. As an extension, some boosted methods analyze content information to further improve the quality of personalized recommendation. However, due to the prevalence of short and sparse messages in online social media, traditional content-boosted methods do not guarantee to capture user preference accurately especially for web contents. In this paper, we propose a novel graphical model to extract hidden topics from web contents, cluster web contents, and detect users' interests on each cluster. In addition, we introduce two reranking models which utilize the detected user interest to further boost the quality of personalized recommendation. Experiment results on a public dataset demonstrated the limitation of a traditional content-boosted approach, and also showed the validity of our proposed techniques.

Due to the revolutionary development of Web 2.0 technology, individual users have become major contributors of Web content in online social media. In light of the growing activities, how to measure a user’s influence to other users in online social media becomes increasingly important. This research need is urgent especially in the online healthcare community since positive influence can be beneficial while negative influence may cause-negative impact on other users of the same community. In this article, a research framework was proposed to study user influence within the online healthcare community. We proposed a new approach to incorporate users’ reply relationship, conversation content and response immediacy which capture both explicit and implicit interaction between users to identify influential users of online healthcare community. A weighted social network is developed to represent the influence between users. We tested our proposed techniques thoroughly on two medical support forums. Two algorithms UserRank and Weighted in-degree are benchmarked with PageRank and in-degree. Experiment results demonstrated the validity and effectiveness of our proposed approaches.

Social network analysis and mining is very useful for law enforcement and intelligence to extract criminals or terrorists interaction patterns and identify their roles in the organizations. Due to the privacy concerns, social network data is usually captured within a law enforcement or intelligence unit without sharing with other units. As a result, the utility of social network analysis is diminished when the social network data within an individual unit is incomplete. In this project, the objectives are sharing the insensitive and generalized information to support social network analysis and mining but preserving the privacy at the same time. We ensure that a prescribed level of privacy leakage tolerance is satisfied. The measurement of the privacy leakage is independent to the privacy preserving techniques of integrating social network data.

Extremists are actively utilizing social media as propaganda to promote their ideologies. Online forums are ideal platforms to draw attention from worldwide Internet users to the timely issues and some opinions in these discussions can be threatening the public safety. It is of great interest for the intelligence to identify clusters on these forums and capture the topics of discussions and their development. Previous work in cluster identification focused on social networks constructed by the direct interactions between users utilizing link analysis techniques. However, the direct interactions between users may only capture one potential relationship between forum users. Users who share common interests may not necessarily interact with each other directly. On the other hand, they may be active in similar events simultaneously. In this paper, we propose a temporal coherence analysis approach to identify clusters of users from the Dark Web data. Users are represented as vectors of activeness and clusters are extracted with the support of temporal coherence analysis. We tested our proposed methods on both synthetic dataset and real world dataset. Using the real-world Dark Web dataset, three clusters were identified and each cluster was also associated with a specific theme. It shows that a cluster of users participating in a theme of discussion can be discovered without using any content analysis but only using temporal analysis.

The four papers in this special issue focus on the advanced modeling and simulation, human organizational interactions, web spidering, digital archiving, cyber archeology, social network analysis, sentiment analysis, and data/text/web mining techniques and methodologies, which can contribute to the understanding of the pulse of the society.

Due to the advancement of Web 2.0 technologies, a large volume of Web opinions is available on social media sites such as Web forums and Weblogs. These technologies provide a platform for Internet users around the world to communicate with each other and express their opinions. Analysis of developing Web opinions is potentially valuable for discovering ongoing topics of interests of the public like terrorist and crime detection, understanding how topics evolve together with the underlying social interaction between participants, and identifying important participants who have great influence in various topics of discussions. Nonetheless, the work of analyzing and clustering Web opinions is extremely challenging. Unlike regular documents, Web opinions are short and sparse text messages with noisy content. Typical document clustering techniques with the goal of clustering all documents applied to Web opinions produce unsatisfactory performance. In this paper, we investigated the density-based clustering algorithm and proposed the scalable distance-based clustering technique for Web opinion clustering. We conducted experiments and benchmarked with the density-based algorithm to show that the new algorithm obtains higher microaccuracy and macroaccuracy. This Web opinion clustering technique enables the identification of themes within discussions in Web social networks and their development, as well as the interactions of active participants. We also developed interactive visualization tools, which make use of the identified topic clusters to display social network development, the network topology similarity between topics, and the similarity values between participants.

As an important information portal, online healthcare forum are playing an increasingly crucial role in disseminating information and offering support to people. It connects people with the leading medical experts and others who have similar experiences. During an epidemic outbreak, such as H1N1, it is critical for the health department to understand how the public is responding to the ongoing pandemic, which has a great impact on the social stability. In this case, identifying influential users in the online healthcare forum and tracking the information spreading in such online community can be an effective way to understand the public reaction toward the disease. In this paper, we propose a framework to monitor and identify influential users from online healthcare forum. We first develop a mechanism to identify and construct social networks from the discussion board of an online healthcare forum. We propose the UserRank algorithm which combines link analysis and content analysis techniques to identify influential users. We have also conducted an experiment to evaluate our approach on the Swine Flu forum which is a sub-community of a popular online healthcare community, MedHelp (www.medhelp.org). Experimental results show that our technique outperforms PageRank, in-degree and out-degree centrality in identifying influential user from an online healthcare forum.

Social network analysis has been shown to be effective in supporting intelligence and law enforcement force to identify suspects, terrorist or criminal subgroups, and their communication patterns. However, social network data owned by individual law enforcement units contain private information that must be preserved before sharing with other law enforcement units. Such privacy issue tremendously reduces the utility of the social network data since the integration of social networks from different law enforcement units cannot be fully integrated. Without integration of social network data, the effectiveness of terrorist or criminal social network analysis is diminished. In this paper, we introduce the KNN and EBB algorithm for constructing generalized subgraphs and a mechanism to integrate the generalized information to conduct the closeness centrality measures. The result shows that the proposed technique improves the accuracy of closeness centrality measures substantially while protecting the sensitive data.

Abstract The category‐tree document‐classification structure is widely used by enterprises and information providers to organize, archive, and access documents for effective knowledge management. However, category trees from various sources use different hierarchical structures, which usually make mappings between categories in different category trees difficult. In this work, we propose a category‐tree integration technique. We develop a method to learn the relationships between any two categories and develop operations such as mapping, splitting, and insertion for this integration. According to the parent‐child relationship of the integrating categories, the developed decision rules use integration operations to integrate categories from the source category tree with those from the master category tree. A unified category tree can accumulate knowledge from multiple resources without forfeiting the knowledge in individual category trees. Experiments have been conducted to measure the performance of the integration operations and the accuracy of the integrated category trees. The proposed category‐tree integration technique achieves greater than 80% integration accuracy, and the insert operation is the most frequently utilized, followed by map and split. The insert operation achieves 77% of F1 while the map and split operations achieves 86% and 29% of F1, respectively.

Looking for web pages to identify useful information from a website is tedious and time consuming. Search engines are not always helpful due to the vocabulary difference between queries and web pages. Users may also have difficulty to accurately represent their information needs as queries at the beginning of exploration stage. A site map of website provides an outline of the overall structure of website. Without navigating through the website from the root page, users can easily identify the exact webpage to extract useful information to satisfy their information needs. However, site maps are not always available. In our previous work, we develop techniques to generate a website topic hierarchy. In this paper, we extend our work to extract keyphrases to label the web site topic hierarchy. The keyphrases serve in the purpose of summarizing the content so that users can efficiently browse through the site map to pin point the web page that provides the useful information they need. In the proposed keyphrase extraction, there are three major components. The first component is the candidate phrases identification. The second component computes the feature scores for summarization. The features include thematic and presentation features. The third component extracts the keyphrases by combining the feature scores. We have conducted an experiment and obtained promising result.

When performing environmental scanning, organizations typically deal with a numerous of events and topics about their core business, relevant technique standards, competitors, and market, where each event or topic to monitor or track generally is associated with many news documents. To reduce information overload and information fatigues when monitoring or tracking such events, it is essential to develop an effective event episode discovery mechanism for organizing all news documents pertaining to an event of interest. In this study, we propose a new metric, referred to as TFxIDFTempo and develop a temporal-based event episode discovery technique that uses the proposed TFxIDFTempo metric as its feature selection method and document representation scheme. Using the traditional TFxIDF-based HAC technique as performance benchmarks, our empirical evaluation results suggest that the proposed temporal-based event episode discovery technique outperforms its benchmark in cluster recall and cluster precision.

Web 2.0 technologies have facilitated the interaction between web users. The business-to-consumer electronic commerce is no longer restricted between consumers and online retail stores. It has been extended to opinions sharing between consumers and consumers. Before making purchasing decision, consumers like to see other consumer opinions in order to identify the best consumer product that fits their preferences. In the recent years, many research works have focused on sentiment classification and analysis of online consumer reviews. Most of them rely on natural language processing techniques to parse and analyze the sentences in online consumer reviews. However, the writing of consumer reviews contributed by web users is usually less formal than those appear in news or journal articles. Many sentences in consumer reviews may contain grammatical errors and unknown terms that do not exist in any dictionaries. As a result, the natural language processing rules are not applicable in many consumer review text and the performance is relatively poor. In this work, we propose to utilize machine learning techniques to classify the consumer product features and produce a summary of consumer reviews for products such as digital cameras. We have conducted an experiment to compare the performance of class association rules and naïve Bayesian classifier for sentiment analysis. The results show that over 70% of macro- and micro- F measures are achieved. It is substantially higher than those achieved by natural language processing approaches.

Due to the popularity of online retail stores, consumers are not only shopping and comparing consumer products on the Web but also providing their consumer reviews on the Internet platform. The Web has become the largest repository of consumer reviews. Consumer reviews are beneficial to consumers, merchants, and manufacturers. Consumers may read the comments of other consumers and decide whether the product is good in the specific product features that they are interested in. For merchants or product manufacturers, consumer reviews help them understand general responses of customers on their products for product or marketing campaign improvement. In addition, consumer reviews can enable merchants better understand specific preferences of individual customers and facilitates effective marketing decisions. However, the large volume of consumer reviews makes it impossible for any individual consumer, merchant, or manufacturer to extract important knowledge efficiently. In this study, we concentrate on opinion sentence identification of focused sentiment analysis and propose a collaborative-filtering-based opinion sentence identification (CF-OSI) technique. The proposed CF-OSI technique considers opinion sentence identification as the sentence retrieval problem. In addition, a collaborative-filtering-based query expansion approach is incorporated into the CF-OSI technique to address possible effectiveness degradation caused by short user queries (i.e., limited number of query terms in query queries). Experiments have been conducted to empirically evaluate the effectiveness of our proposed technique. Our evaluation results show that the performance of our proposed CF-OSI technique is promising.

We propose a ranking model to diversify answers of non-factoid questions based on an inverse notion of graph connectivity. By representing a collection of candidate answers as a graph, we posit that novelty, a measure of diversity, is inversely proportional to answer vertices' connectivity. Hence, unlike the typical graph ranking models, which score vertices based on the degree of connectedness, our method assigns a penalty score for a candidate answer if it is strongly connected to other answers. That is, any redundant answers, indicated by a higher inter-sentence similarity, will be ranked lower than those with lower inter-sentence similarity. At the end of the ranking iterations, many redundant answers will be moved toward the bottom on the ranked list. The experimental results show that our method helps diversify answer coverage of non-factoid questions according to F-scores from nugget pyramid evaluation.

Food safety issues often draw public attention after the discovery of suspected or confirmed cases of food poisoning and contamination. Food safety incidents reveal voids in existing food safety practices established by food science, governmental policies, and business processes. The vulnerability in food supply chain may yield room for potential food terrorism. This paper presents a framework for harnessing public wisdom from mass media to improve and advance existing practices to better ensure our food safety. The framework incorporates the use of content and social network analyses to distill important issues into food safety knowledge.

Social networks are valuable resources for intelligence and law enforcement force in their investigations when they want to identify suspects, terrorist or criminal subgroups and their communication patterns. However, missing information in a terrorist or criminal social network always diminish the effectiveness of investigation. Sharing and integration of social networks from different agencies helps increasing its effectiveness; however, information sharing is usually forbidden due to the concern of privacy protection. In this paper, we introduce the subgraph generalization and mechanism to integrate generalized information to conduct social network analysis.

Due to the advance of Web 2.0 technologies, a large volume of Web opinions are available in computer-mediated communication sites such as forums and blogs. Many of these Web opinions involve terrorism and crime related issues. For instances, some terrorist groups may use Web forums to propagandize their ideology, some may post threaten messages, and some criminals may recruit members or identify victims through Web social networks. Analyzing and clustering Web opinions are extremely challenging. Unlike regular documents, Web opinions usually appear as short and sparse text messages. Using typical document clustering techniques on Web opinions produce unsatisfying result. In this work, we propose the scalable distance-based clustering technique for Web opinions clustering. We have conducted experiments and benchmarked with the density-based algorithm. It shows that it obtains higher micro and macro accuracy. This Web opinions clustering technique is useful in identifying the themes of discussions in Web social networks and studying their development as well as the interactions of active participants.

Abstract Navigating through hyperlinks within a Web site to look for information from one of its Web pages without the support of a site map can be inefficient and ineffective. Although the content of a Web site is usually organized with an inherent structure like a topic hierarchy, which is a directed tree rooted at a Web site's homepage whose vertices and edges correspond to Web pages and hyperlinks, such a topic hierarchy is not always available to the user. In this work, we studied the problem of automatic generation of Web sites' topic hierarchies. We modeled a Web site's link structure as a weighted directed graph and proposed methods for estimating edge weights based on eight types of features and three learning algorithms, namely decision trees, naïve Bayes classifiers, and logistic regression. Three graph algorithms, namely breadth‐first search, shortest‐path search, and directed minimum‐spanning tree, were adapted to generate the topic hierarchy based on the graph model. We have tested the model and algorithms on real Web sites. It is found that the directed minimum‐spanning tree algorithm with the decision tree as the weight learning algorithm achieves the highest performance with an average accuracy of 91.9%.

Given the advance of Internet technologies, we can now easily extract hundreds or thousands of news stories of any ongoing incidents from newswires such as CNN.com, but the volume of information is too large for us to capture the blueprint. Information retrieval techniques such as topic detection and tracking are able to organize news stories as events, in a flat hierarchical structure, within a topic. However, they are incapable of presenting the complex evolution relationships between the events. We are interested to learn not only what the major events are but also how they develop within the topic. It is beneficial to identify the seminal events, the intermediary and ending events, and the evolution of these events. In this paper, we propose to utilize the event timestamp, event content similarity, temporal proximity, and document distributional proximity to model the event evolution relationships between events in an incident. An event evolution graph is constructed to present the underlying structure of events for efficient browsing and extracting of information. Case study and experiments are presented to illustrate and show the performance of our proposed technique. It is found that our proposed technique outperforms the baseline technique and other comparable techniques in previous work.

Terrorist or criminal social network analysis is helpful for intelligence and law enforcement force in investigation. However, individual agency usually has part of the complete terrorist or criminal social network and therefore some crucial knowledge is not able to be extracted. Sharing information between different agencies will make such social network analysis more effective; unfortunately, it may violate the privacy of some sensitive information. There is always a tradeoff between the degree of privacy and the degree of utility in information sharing. Several approaches have been proposed to resolve such dilemma in sharing data from different relational tables. There is not any work on sharing social networks from different sources and yet try to minimize the reduction on the degree of privacy. In this paper, we propose a subgraph generalization approach for information sharing and privacy protection of terrorist or criminal social networks. Our experiment shows that such approach is promising.

Web forums provide platforms for any Internet users around the world to communicate with each other and express their opinions. In many of the discussions in Web forums, it involves issues related to terrorism and crime. Some participants are even using the platform to propagandize their ideology or recruit members to commit crime. In this work, we propose a Web forum analysis system to analyze the content development and visualize the social interactions in Web forum.

Abstract This article analyzes the collaboration trends, authorship and keywords of all research articles published in the Journal of American Society for Information Science and Technology (JASIST). Comparing the articles between two 10‐year periods, namely, 1988–1997 and 1998–2007, the three‐fold objectives are to analyze the shifts in (a) authors' collaboration trends (b) top authors, their affiliations as well as the pattern of coauthorship among them, and (c) top keywords and the subdisciplines from which they emerge. The findings reveal a distinct tendency towards collaboration among authors, with external collaborations becoming more prevalent. Top authors have grown in diversity from those being affiliated predominantly with library/information‐related departments to include those from information systems management, information technology, businesss, and the humanities. Amid heterogeneous clusters of collaboration among top authors, strongly connected cross‐disciplinary coauthor pairs have become more prevalent. Correspondingly, the distribution of top keywords' occurrences that leans heavily on core information science has shifted towards other subdisciplines such as information technology and sociobehavioral science.

Abstract Many automatic text summarization models have been developed in the last decades. Related research in information science has shown that human abstractors extract sentences for summaries based on the hierarchical structure of documents; however, the existing automatic summarization models do not take into account the human abstractor's behavior of sentence extraction and only consider the document as a sequence of sentences during the process of extraction of sentences as a summary. In general, a document exhibits a well‐defined hierarchical structure that can be described as fractals—mathematical objects with a high degree of redundancy. In this article, we introduce the fractal summarization model based on the fractal theory. The important information is captured from the source document by exploring the hierarchical structure and salient features of the document. A condensed version of the document that is informatively close to the source document is produced iteratively using the contractive transformation in the fractal theory. The fractal summarization model is the first attempt to apply fractal theory to document summarization. It significantly improves the divergence of information coverage of summary and the precision of summary. User evaluations have been conducted. Results have indicated that fractal summarization is promising and outperforms current summarization techniques that do not consider the hierarchical structure of documents.

A Weblog is a Web site where entries are made in diary style, maintained by its sole author - a blogger, and displayed in a reverse chronological order. Due to the freedom and convenience of publishing in Weblogs, this form of media provides an ideal environment as a propaganda platform for terrorist groups to promote their ideologies and as an operation platform for organizing crimes. In this work, we present a framework to analyze and visualize Weblog social network embedded beneath relevant Weblogs gathered through topic-specific exploration. Link analysis uses the relationships between bloggers to construct the Weblog social network. Content analysis associates similar blog messages to unveil implicit relationships found in the semantics to further improve the Weblog social network analysis. Users can use different interactive information visualization techniques to explore various aspects of the underlying social network at different levels of abstraction. With the capability of analyzing and visualizing Weblog social networks in terrorist and crime related matters, intelligence agencies and law enforcement will be able to have an additional tools and means to ensure the national security.

Hierarchical models are commonly used to organize a Website's content. A Website's content structure can be represented by a topic hierarchy, a directed tree rooted at a Website's homepage in which the vertices and edges correspond to Web pages and hyperlinks. In this work, we propose a new method for constructing the topic hierarchy of a Website. We model the Website's link structure using weighted directed graph, in which the edge weights are computed using a classifier that predicts if an edge connects a pair of nodes representing a topic and a sub-topic. We then pose the problem of building the topic hierarchy as finding the shortest-path tree and directed minimum spanning tree in the weighted graph. We've done extensive experiments using real Websites and obtained very promising results.

Documents in the Web are often organized using category trees by information providers (e.g. CNN, BBC) or search engines (e.g. Google, Yahoo!). Such category trees are commonly known as Web directories. The category tree structures from different internet content providers may be similar to some extent but are usually not exactly the same. As a result, it is desirable to integrate these category trees together so that web users only need to browse through a unified category tree to extract information from multiple providers. In this paper, we address this problem by capturing structural information of multiple category trees, which are embedded with the knowledge of professional in organizing the documents. Our experiments with real Web data show that the proposed technique is promising.

Abstract The number of non‐English resources has been increasing rapidly on the Web. Although many studies have been conducted on the query logs in search engines that are primarily English‐based (e.g., Excite and AltaVista), only a few of them have studied the information‐seeking behavior on the Web in non‐English languages. In this article, we report the analysis of the search‐query logs of a search engine that focused on Chinese. Three months of search‐query logs of Timway, a search engine based in Hong Kong, were collected and analyzed. Metrics on sessions, queries, search topics, and character usage are reported. N‐gram analysis also has been applied to perform character‐based analysis. Our analysis suggests that some characteristics identified in the search log, such as search topics and the mean number of queries per sessions, are similar to those in English search engines; however, other characteristics, such as the use of operators in query formulation, are significantly different. The analysis also shows that only a very small number of unique Chinese characters are used in search queries. We believe the findings from this study have provided some insights into further research in non‐English Web searching.

Abstract The amount of information on the Web has been expanding at an enormous pace. There are a variety of Web documents in different genres, such as news, reports, reviews. Traditionally, the information displayed on Web sites has been static. Recently, there are many Web sites offering content that is dynamically generated and frequently updated. It is also common for Web sites to contain information in different languages since many countries adopt more than one language. Moreover, content may exist in multimedia formats including text, images, video, and audio.

Abstract With the overwhelming volume of information, the task of finding relevant information on a given topic on the Web is becoming increasingly difficult. Web search engines hence become one of the most popular solutions available on the Web. However, it has never been easy for novice users to organize and represent their information needs using simple queries. Users have to keep modifying their input queries until they get expected results. Therefore, it is often desirable for search engines to give suggestions on related queries to users. Besides, by identifying those related queries, search engines can potentially perform optimizations on their systems, such as query expansion and file indexing. In this work we propose a method that suggests a list of related queries given an initial input query. The related queries are based in the query log of previously submitted queries by human users, which can be identified using an enhanced model of association rules. Users can utilize the suggested related queries to tune or redirect the search process. Our method not only discovers the related queries, but also ranks them according to the degree of their relatedness. Unlike many other rival techniques, it also performs reasonably well on less frequent input queries.

Abstract Modern information retrieval systems use keywords within documents as indexing terms for search of relevant documents. As Chinese is an ideographic character‐based language, the words in the texts are not delimited by white spaces. Indexing of Chinese documents is impossible without a proper segmentation algorithm. Many Chinese segmentation algorithms have been proposed in the past. Traditional segmentation algorithms cannot operate without a large dictionary or a large corpus of training data. Nowadays, the Web has become the largest corpus that is ideal for Chinese segmentation. Although most search engines have problems in segmenting texts into proper words, they maintain huge databases of documents and frequencies of character sequences in the documents. Their databases are important potential resources for segmentation. In this paper, we propose a segmentation algorithm by mining Web data with the help of search engines. On the other hand, the Romanized pinyin of Chinese language indicates boundaries of words in the text. Our algorithm is the first to utilize the Romanized pinyin to segmentation. It is the first unified segmentation algorithm for the Chinese language from different geographical areas, and it is also domain independent because of the nature of the Web. Experiments have been conducted on the datasets of a recent Chinese segmentation competition. The results show that our algorithm outperforms the traditional algorithms in terms of precision and recall. Moreover, our algorithm can effectively deal with the problems of segmentation ambiguity, new word (unknown word) detection, and stop words.

In this work, we study similarity measures for text-centric XML documents based on an extended vector space model, which considers both document content and structure. Experimental results based on a benchmark showed superior performance of the proposed measure over the baseline which ignores structural knowledge of XML documents.

Information extraction is important for crime analysis. Due to the popularity of the Web, information related to crime and terrorism is available in multiple languages. As a result, cross-lingual semantic interoperability is essential when we extract information across multiple languages. In our previous work, we have developed several techniques to generate an automatic cross-lingual thesaurus to support cross-lingual information retrieval based on a parallel corpus collected from the Web. The techniques include Hopfield network and associate constraint network with backmarking. Although these techniques obtain satisfactory performance, they have weaknesses in efficiency, consistency, precision or recall. In this work, we develop a new searching technique, namely forward evaluation, on the basis of our previously developed associate constraint network model. We have conducted an experiment and show that the proposed forward evaluation technique outperforms both Hopfield network and associate constraint network with backmarking in terms of precision and recall. In addition, its efficiency is better than Hopfield network but is not as good as associate constraint network with backmarking.

In this work we propose a method that retrieves a list of related queries given an initial input query. The related queries are based on the query log of previously issued queries by human users, which can be discovered using our improved association rule mining model. Users can use the suggested related queries to tune or redirect the search process. Our method not only discovers the related queries, but also ranks them according to the degree of their relatedness. Unlike many other rival techniques, it exploits only limited query log information and performs relatively better on queries in all frequency divisions.

In this paper, we propose an approach to automatically mine event evolution graphs from newswires on the Web. Event evolution graph is a directed graph in which the vertices and edges denote news events and the evolutions between events respectively, in a news affair. Our model utilizes the content similarity between events and incorporates temporal proximity and document distributional proximity as decaying functions. Our approach is effective in presenting the inside developments of news affairs along the timeline, which can facilitate users' information browsing tasks.

Abstract As illustrated by the World Wide Web, the volume of information in languages other than English has grown significantly in recent years. This highlights the importance of multilingual corpora. Much effort has been devoted to the compilation of multilingual corpora for the purpose of cross‐lingual information retrieval and machine translation. Existing parallel corpora mostly involve European languages, such as English–French and English–Spanish. There is still a lack of parallel corpora between European languages and Asian languages. In the authors' previous work, an alignment method to identify one‐to‐one Chinese and English title pairs was developed to construct an English–Chinese parallel corpus that works automatically from the World Wide Web, and a 100% precision and 87% recall were obtained. Careful analysis of these results has helped the authors to understand how the alignment method can be improved. A conceptual analysis was conducted, which includes the analysis of conceptual equivalent and conceptual information alternation in the aligned and nonaligned English–Chinese title pairs that are obtained by the alignment method. The result of the analysis not only reflects the characteristics of parallel corpora, but also gives insight into the strengths and weaknesses of the alignment method. In particular, conceptual alternation, such as omission and addition, is found to have a significant impact on the performance of the alignment method.

Abstract Based on the salient features of the documents, automatic text summarization systems extract the key sentences from source documents. This process supports the users in evaluating the relevance of the extracted documents returned by information retrieval systems. Because of this tool, efficient filtering can be achieved. Indirectly, these systems help to resolve the problem of information overloading. Many automatic text summarization systems have been implemented for use with different languages. It has been established that the grammatical and lexical differences between languages have a significant effect on text processing. However, the impact of the language differences on the automatic text summarization systems has not yet been investigated. The authors provide an impact analysis of language difference on automatic text summarization. It includes the effect on the extraction processes, the scoring mechanisms, the performance, and the matching of the extracted sentences, using the parallel corpus in English and Chinese as the tested object. The analysis results provide a greater understanding of language differences and promote the future development of more advanced text summarization techniques.

Hierarchical models are commonly used to organize a Website's content. A Website's content structure can be represented by a topic hierarchy, a directed tree rooted at a Website's homepage in which the vertices and edges correspond to Web pages and hyperlinks. In this work, we propose an algorithm for extracting a Website's topic hierarchy from its link structure. The proposed algorithm consists of a construction stage and a refining stage, in which we analyze the semantic relationships between web pages based on link structure, web page content and directory structure. We've done extensive experiments using different Websites and obtained very promising results.

Abstract For the sake of national security, very large volumes of data and information are generated and gathered daily. Much of this data and information is written in different languages, stored in different locations, and may be seemingly unconnected. Crosslingual semantic interoperability is a major challenge to generate an overview of this disparate data and information so that it can be analyzed, shared, searched, and summarized. The recent terrorist attacks and the tragic events of September 11, 2001 have prompted increased attention on national security and criminal analysis. Many Asian countries and cities, such as Japan, Taiwan, and Singapore, have been advised that they may become the next targets of terrorist attacks. Semantic interoperability has been a focus in digital library research. Traditional information retrieval (IR) approaches normally require a document to share some common keywords with the query. Generating the associations for the related terms between the two term spaces of users and documents is an important issue. The problem can be viewed as the creation of a thesaurus. Apart from this, terrorists and criminals may communicate through letters, e‐mails, and faxes in languages other than English. The translation ambiguity significantly exacerbates the retrieval problem. The problem is expanded to crosslingual semantic interoperability. In this paper, we focus on the English/Chinese crosslingual semantic interoperability problem. However, the developed techniques are not limited to English and Chinese languages but can be applied to many other languages. English and Chinese are popular languages in the Asian region. Much information about national security or crime is communicated in these languages. An efficient automatically generated thesaurus between these languages is important to crosslingual information retrieval between English and Chinese languages. To facilitate crosslingual information retrieval, a corpus‐based approach uses the term co‐occurrence statistics in parallel or comparable corpora to construct a statistical translation model to cross the language boundary. In this paper, the text‐based approach to align English/Chinese Hong Kong Police press release documents from the Web is first presented. We also introduce an algorithmic approach to generate a robust knowledge base based on statistical correlation analysis of the semantics (knowledge) embedded in the bilingual press release corpus. The research output consisted of a thesaurus‐like, semantic network knowledge base, which can aid in semantics‐based crosslingual information management and retrieval.

Abstract The authors propose a heuristic method for Chinese automatic text segmentation based on a statistical approach. This method is developed based on statistical information about the association among adjacent characters in Chinese text. Mutual information of bi‐grams and significant estimation of tri‐grams are utilized. A heuristic method with six rules is then proposed to determine the segmentation points in a Chinese sentence. No dictionary is required in this method. Chinese text segmentation is important in Chinese text indexing and thus greatly affects the performance of Chinese information retrieval. Due to the lack of delimiters of words in Chinese text, Chinese text segmentation is more difficult than English text segmentation. Besides, segmentation ambiguities and occurrences of out‐of‐vocabulary words (i.e., unknown words) are the major challenges in Chinese segmentation. Many research studies dealing with the problem of word segmentation have focused on the resolution of segmentation ambiguities. The problem of unknown word identification has not drawn much attention. The experimental result shows that the proposed heuristic method is promising to segment the unknown words as well as the known words. The authors further investigated the distribution of the errors of commission and the errors of omission caused by the proposed heuristic method and benchmarked the proposed heuristic method with a previous proposed technique, boundary detection. It is found that the heuristic method outperformed the boundary detection method.

Abstract Peer‐to‐Peer (P2P) networks provide a new distributed computing paradigm on the Internet for file sharing. The decentralized nature of P2P networks fosters cooperative and non‐cooperative behaviors in sharing resources. Searching is a major component of P2P file sharing. Several studies have been reported on the nature of queries of World Wide Web (WWW) search engines, but studies on queries of P2P networks have not been reported yet. In this report, we present our study on the Gnutella network, a decentralized and unstructured P2P network. We found that the majority of Gnutella users are located in the United States. Most queries are repeated. This may be because the hosts of the target files connect or disconnect from the network any time, so clients resubmit their queries. Queries are also forwarded from peers to peers. Findings are compared with the data from two other studies of Web queries. The length of queries in the Gnutella network is longer than those reported in the studies of WWW search engines. Queries with the highest frequency are mostly related to the names of movies, songs, artists, singers, and directors. Terms with the highest frequency are related to file formats, entertainment, and sexuality. This study is important for the future design of applications, architecture, and services of P2P networks.

In this paper, we introduce the fractal summarization model based on the fractal theory. In fractal summarization, the important information is captured from the source text by exploring the hierarchical structure and salient features of the document. A condensed version of the document that is informatively close to the original is produced iteratively using the contractive transformation in the fractal theory. User evaluation has shown that fractal summarization outperforms traditional summarization.

Wireless access with mobile (or handheld) devices is a promising addition to the WWW and traditional electronic business. Mobile devices provide convenience and portable access to the huge information space on the Internet without requiring users to be stationary with network connection. However, the limited screen size, narrow network bandwidth, small memory capacity and low computing power are the shortcomings of handheld devices. Loading and visualizing large documents on handheld devices become impossible. The limited resolution restricts the amount of information to be displayed. The download time is intolerably long. In this paper, we introduce the fractal summarization model for document summarization on handheld devices. Fractal summarization is developed based on the fractal theory. It generates a brief skeleton of summary at the first stage, and the details of the summary on different levels of the document are generated on demands of users. Such interactive summarization reduces the computation load in comparing with the generation of the entire summary in one batch by the traditional automatic summarization, which is ideal for wireless access. Three-tier architecture with the middle-tier conducting the major computation is also discussed. Visualization of summary on handheld devices is also investigated.

Abstract The information available in languages other than English in the World Wide Web is increasing significantly. According to a report from Computer Economics in 1999, 54% of Internet users are English speakers (“English Will Dominate Web for Only Three More Years,”Computer Economics, July 9, 1999 , http://www.computereconomics.com/new4/pr/pr990610.html ). However, it is predicted that there will be only 60% increase in Internet users among English speakers verses a 150% growth among non‐English speakers for the next five years. By 2005, 57% of Internet users will be non‐English speakers. A report by CNN.com in 2000 showed that the number of Internet users in China had been increased from 8.9 million to 16.9 million from January to June in 2000 (“Report: China Internet users double to 17 million,”CNN.com, July, 2000 , http://cnn.org/2000/TECH/computing/07/27/china.internet.reut/index.html ). According to Nielsen/NetRatings, there was a dramatic leap from 22.5 millions to 56.6 millions Internet users from 2001 to 2002. China had become the second largest global at‐home Internet population in 2002 (US's Internet population was 166 millions) (Robyn Greenspan, “China Pulls Ahead of Japan,” Internet.com, April 22, 2002 , http://cyberatlas.internet.com/big_picture/geographics/article/0,,5911_1013841,00.html ). All of the evidences reveal the importance of cross‐lingual research to satisfy the needs in the near future. Digital library research has been focusing in structural and semantic interoperability in the past. Searching and retrieving objects across variations in protocols, formats and disciplines are widely explored (Schatz, B., & Chen, H. ( 1999 ). Digital libraries: technological advances and social impacts. IEEE Computer, Special Issue on Digital Libraries, February, 32(2), 45–50.; Chen, H., Yen, J., & Yang, C.C. ( 1999 ). International activities: development of Asian digital libraries. IEEE Computer, Special Issue on Digital Libraries, 32(2), 48–49.). However, research in crossing language boundaries, especially across European languages and Oriental languages, is still in the initial stage. In this proposal, we put our focus oncross‐lingual semantic interoperabilityby developing automatic generation of a cross‐lingual thesaurus based on English/Chinese parallel corpus. When the searchers encounter retrieval problems, professional librarians usually consult the thesaurus to identify other relevant vocabularies. In the problem of searching across language boundaries, a cross‐lingual thesaurus, which is generated by co‐occurrence analysis and Hopfield network, can be used to generate additional semantically relevant terms that cannot be obtained from dictionary. In particular, the automatically generated cross‐lingual thesaurus is able to capture the unknown words that do not exist in a dictionary, such as names of persons, organizations, and events. Due to Hong Kong's unique history background, both English and Chinese are used as official languages in all legal documents. Therefore, English/Chinese cross‐lingual information retrieval is critical for applications in courts and the government. In this paper, we develop an automatic thesaurus by the Hopfield network based on a parallel corpus collected from the Web site of the Department of Justice of the Hong Kong Special Administrative Region (HKSAR) Government. Experiments are conducted to measure the precision and recall of the automatic generated English/Chinese thesaurus. The result shows that such thesaurus is a promising tool to retrieve relevant terms, especially in the language that is not the same as the input term. The direct translation of the input term can also be retrieved in most of the cases.

Abstract As the demand for global information increases significantly, multilingual corpora has become a valuable linguistic resource for applications to cross‐lingual information retrieval and natural language processing. In order to cross the boundaries that exist between different languages, dictionaries are the most typical tools. However, the general‐purpose dictionary is less sensitive in both genre and domain. It is also impractical to manually construct tailored bilingual dictionaries or sophisticated multilingual thesauri for large applications. Corpus‐based approaches, which do not have the limitation of dictionaries, provide a statistical translation model with which to cross the language boundary. There are many domain‐specific parallel or comparable corpora that are employed in machine translation and cross‐lingual information retrieval. Most of these are corpora between Indo‐European languages, such as English/French and English/Spanish. The Asian/Indo‐European corpus, especially English/Chinese corpus, is relatively sparse. The objective of the present research is to construct English/Chinese parallel corpus automatically from the World Wide Web. In this paper, an alignment method is presented which is based on dynamic programming to identify the one‐to‐one Chinese and English title pairs. The method includes alignment at title level, word level and character level. The longest common subsequence (LCS) is applied to find the most reliable Chinese translation of an English word. As one word for a language may translate into two or more words repetitively in another language, the edit operation, deletion, is used to resolve redundancy. A score function is then proposed to determine the optimal title pairs. Experiments have been conducted to investigate the performance of the proposed method using the daily press release articles by the Hong Kong SAR government as the test bed. The precision of the result is 0.998 while the recall is 0.806. The release articles and speech articles, published by Hongkong & Shanghai Banking Corporation Limited, are also used to test our method, the precision is 1.00, and the recall is 0.948.

Abstract As the Web has become a major channel of information dissemination, many newspapers expand their services by providing electronic versions of news information on the Web. However, most investors find it difficult to search for the financial information of interest from the huge Web information space–information overloading problem. In this article, we present a personal agent that utilizes user profiles and user relevance feedback to search for the Chinese Web financial news articles on behalf of users. A Chinese indexing component is developed to index the continuously fetched Chinese financial news articles. User profiles capture the basic knowledge of user preferences based on the sources of news articles, the regions of the news reported, categories of industries related, the listed companies, and user‐specified keywords. User feedback captures the semantics of the user rated news articles. The search engine ranks the top 20 news articles that users are most interested in and report to the user daily or on demand. Experiments are conducted to measure the performance of the agents based on the inputs from user profiles and user feedback. It shows that simply using the user profiles does not increase the precision of the retrieval. However, user relevance feedback helps to increase the performance of the retrieval as the user interact with the system until it reaches the optimal performance. Combining both user profiles and user relevance feedback produces the best performance.

Multiple color coordinate systems are usually involved in color image applications or systems. Forward and backward transformations are used to switch between color coordinate systems. The out of gamut problem emerges when the color coordinate systems' gamut is different. Conventional approaches, including clipping R, G, B values and clipping luminance value are not ideal as they both produce severe error in the luminance component of the resulting color or reduce the contrast of the resulting image. We propose to clip the saturation value when a vector is out of the gamut of the LHS or YIQ space using the saturation processing equations. We have conducted experiments to compare three different clipping approaches. The experimental results show that the saturation clipping approach outperforms the other two approaches, in the sense that it can keep the luminance value unchanged. In addition, the contrast of the image produced by saturation clipping is much better than those produced by other clipping approaches.

Digital libraries store materials in electronic format. Research and development in digital libraries includes content creation, conversion, indexing, organization, and dissemination. The key technological issues are how to search and display desired selections from and across large collections effectively [Schatz & Chen, 1996]. Digital library research projects (DLI-1) sponsored by NSF/DARPA/NASA have a common theme of bringing search to the net, which is the flagship research effort for the National Information Infrastructure (NII) in the United States. A repository is an indexed collection of objects. Indexing is an important task for searching. The better the indexing, the better the searching result. Developing a universal digital library has been the dream of many researchers, however, there are still many problems to be solved before such a vision is fulfilled. The most critical is to support a cross-lingual retrieval or multilingual digital library. Much work has been done on English information retrieval, however, there is relatively less work on Chinese information retrieval. In this article, we focus on Chinese indexing, which is the foundation of Chinese and cross-lingual information retrieval. The smallest indexing units in Chinese digital libraries are words, while the smallest units in a Chinese sentence are characters. However, Chinese text has no delimiter to mark word boundaries as it is in English text. In English or other languages using Roman or Greek-based orthographies, often, spacing reliably indicates word boundaries. In Chinese, a number of characters are placed together without any delimiters indicating the boundaries between consecutive characters. In this article, we investigate the combination and boundary detection approaches based on mutual information for segmentation. The combination approach combines n-grams to form words with more number of characters. In the combination approach Algorithm 1 does not allow overlapping of n-grams while Algorithm 2 does. The boundary detection approach detects the segmentation points on a sentence based on the values and the change of values of the mutual information. Experiments are conducted to evaluate their performances. An interface of the system is also presented to show how a Chinese web page is downloaded, the text in the page filtered, and segmented into words. The segmented words can be submitted for indexing or new unknown words can be identified and submitted to a dictionary.

As part of the ongoing Illinois Digital Library Initiative project, this research proposes an intelligent agent approach to Web searching. In this experiment, we developed two Web personal spiders based on best first search and genetic algorithm techniques, respectively. These personal spiders can dynamically take a user's selected starting homepages and search for the most closely related homepages in the Web, based on the links and keyword indexing. A graphical, dynamic, Java-based interface was developed and is available for Web access. A system architecture for implementing such an agent-based spider is presented, followed by detailed discussions of benchmark testing and user evaluation results. In benchmark testing, although the genetic algorithm spider did not outperform the best first search spider, we found both results to be comparable and complementary. In user evaluation, the genetic algorithm spider obtained significantly higher recall value than that of the best first search spider. However, their precision values were not statistically different. The mutation process introduced in genetic algorithm allows users to find other potential relevant homepages that cannot be explored via a conventional local search process. In addition, we found the Java-based interface to be a necessary component for design of a truly interactive and dynamic Web agent. © 1998 John Wiley & Sons, Inc.

This paper discusses the effect of spatial quantization errors and displacement errors on the precision dimensional measurements for an edge segment. Probabilistic analysis in terms of the resolution of the image is developed for 2D quantization errors. Expressions for the mean and variance of these errors are developed. The probability density function of the quantization error is derived. The position and orientation errors of the active head are assumed to be normally distributed. A probabilistic analysis in terms of these errors is developed for the displacement errors. Through integrating the spatial quantization errors and the displacement errors, we can compute the total error in the active vision inspection system. Based on the developed analysis, we investigate whether a given set of sensor setting parameters in an active system is suitable to obtain a desired accuracy for specific dimensional measurements, and one can determine sensor positions and view directions which meet the necessary tolerance and accuracy of inspection.

An important aspect of inspection planning involves determining camera poses based on some criterion. We seek to find camera poses where the effects of displacement and quantization errors are minimal. The mean squared error is formulated, including all dependencies, and minimized to determine an optimal camera pose that satisfies the sensor constraints of resolution, focus, field-of-view, and visibility. Dimensional tolerances for line entities are also formulated and exploited to determine the acceptability of a given camera pose for all entities observed.

This paper proposes interval constraint network and interval propagation techniques for automatic tolerance design. A hierarchical representation is utilized in the interval constraint network. The consistency of a constraint is defined for the purpose of tolerance design. Forward and backward propagation techniques are introduced in the interval constraint network for tolerance analysis and synthesis, respectively. Both a propagation technique for a single constraint and a parallel propagation technique for multiple constraints between two adjacent levels in the network are introduced. Experiments conducted to illustrate the procedures of tolerance analysis and synthesis for the tank problem are described.

Displacement error is inherent in automated visual inspection systems. This paper discusses the effect of displacement error of the end-effector on the precision measurement of the dimension of an edge line segment. The position and orientation errors of the end-effector are assumed to be normal distributed. A probabilistic analysis in terms of the these errors is developed for the displacement errors. Given that the nominal orientation and position of the sensor are known, the effect of this error on the dimension measurement is also analyzed. Based on this analysis, we investigate whether a given set of sensor setting parameters in an active system is suitable to obtain a desired accuracy for specific line segment dimensional measurements. In addition, based on this approach, one can determine sensor positions and view directions which meet specific targets for tolerance and accuracy of inspection. Developing these mechanisms is central to achieving effective, economic, and accurate inspection systems.

This paper is concerned with problems in automated visual inspection of manufactured (particularly machined) components based on their (CAD) design models. In order to achieve the integrated intelligent inspection goals, the authors address several interrelated problems. These problems include: (i) developing hierarchical representation mechanisms to effectively capture the knowledge about geometric entities, their relationships, sensors, and plans, (ii) reasoning mechanisms to determine the different attributes of the different features of an object to be inspected, and the alternative strategies which can be used for inspection of each attribute, (iii) strategies for automated generation of position and viewing angles of the cameras in an active vision system, and for determining the visible entities in each configuration, and (iv) optimization of the constructed plan including minimizing the number of sensor settings and the total distance traveled by an active visual sensor.>

A common approach to color image processing is to apply monochrome techniques to the quantized intensity component. However, the quantization error in the intermediate intensity image propagates to the final processed image. This can lead to significant distortion, depending on the input RGB values and on the particular image processing function being applied. A theoretical analysis of the worst-case quantization error is presented. Experimental results with histogram equalization demonstrate how the histogram resolution affects the performance.