Fact verification is extremely challenging in natural language processing tasks, requiring the retrieval of multiple evidence sentences from trustworthy corpora to ascertain the accuracy of a given claim. Although the current methods have achieved satisfactory performance, many of them ignore multi-granularity information or fail to fully leverage multi-granularity information, and lack inherent concept information. To tackle the issues, we propose the Concept-Enhanced Heterogeneous Graph Network (Concept-HGN) for fact verification. First, our Concept-HGN model constructs a heterogeneous graph to aggregate clues from the scattered text across multiple evidence sentences. By building different heterogeneous nodes into an integral unified graph, this hierarchical node granularity enables Concept-HGN to be more effectively applied to fact verification tasks. Then, Concept-HGN leverages the intrinsic concepts of entities from YAGO, guiding fact verification and boosting the fact verification performance. We conducted performance evaluation experiments on the FEVER and UKP Snopes datasets. On the FEVER dataset, our proposed Concept-HGN model achieved 80.26 % and 77.68 % on LA and FS, respectively. On the UKP Snopes dataset, the accuracy and macro F1 also reached 65.7 % and 61.9 %, respectively. The experimental results on these datasets indicate that the Concept-HGN model proposed in this paper outperforms the baseline models and achieves state-of-the-art performance on the task of fact verification.

Large language models (LLMs) have demonstrated impressive performance across various domains. However, for clinical diagnosis, higher expectations are required for LLM’s reliability and sensitivity: thinking like physicians and remaining sensitive to key medical information that affects diagnostic reasoning, as subtle variations can lead to different diagnosis results. Yet, existing works focus mainly on investigating the sensitivity of LLMs to irrelevant context and overlook the importance of key information. In this paper, we investigate the sensitivity of LLMs, i.e. GPT-3.5, GPT-4, Gemini, Claude3 and LLaMA2-7b, to key medical information by introducing different perturbation strategies. The evaluation results highlight the limitations of current LLMs in remaining sensitive to key medical information for diagnostic decision-making. The evolution of LLMs must focus on improving their reliability, enhancing their ability to be sensitive to key information, and effectively utilizing this information. These improvements will enhance human trust in LLMs and facilitate their practical application in real-world scenarios. Our code and dataset are available at https://github.com/chenwei23333/DiagnosisQA.

Many question-answering problems can be approached as textual entailment tasks, where the hypotheses are formed by the question and candidate answers, and the premises are derived from an external knowledge base. However, current neural methods often lack transparency in their decision-making processes. Moreover, first-order logic methods, while systematic, struggle to integrate unstructured external knowledge. To address these limitations, we propose a neuro-symbolic reasoning framework calledFinal, which combinesFIrst-order logic withNAturalLogic for question answering. Our framework utilizesfirst-order logicto systematically decompose hypotheses andnatural logicto construct reasoning paths from premises to hypotheses, employing bidirectional reasoning to establish links along the reasoning path. This approach not only enhances interpretability but also effectively integrates unstructured knowledge. Our experiments on three benchmark datasets, namely QASC, WorldTree, and WikiHop, demonstrate thatFinaloutperforms existing methods in commonsense reasoning and reading comprehension tasks, achieving state-of-the-art results. Additionally, our framework also provides transparent reasoning paths that elucidate the rationale behind the correct decisions.

BACKGROUND: Detecting event triggers in biomedical texts, which contain domain knowledge and context-dependent terms, is more challenging than in general-domain texts. Most state-of-the-art models rely mainly on external resources such as linguistic tools and knowledge bases to improve system performance. However, they lack effective mechanisms to obtain semantic clues from label specification and sentence context. Given its success in image classification, label representation learning is a promising approach to enhancing biomedical event trigger detection models by leveraging the rich semantics of pre-defined event type labels. RESULTS: In this paper, we propose the Biomedical Label-based Synergistic representation Learning (BioLSL) model, which effectively utilizes event type labels by learning their correlation with trigger words and enriches the representation contextually. The BioLSL model consists of three modules. Firstly, the Domain-specific Joint Encoding module employs a transformer-based, domain-specific pre-trained architecture to jointly encode input sentences and pre-defined event type labels. Secondly, the Label-based Synergistic Representation Learning module learns the semantic relationships between input texts and event type labels, and generates a Label-Trigger Aware Representation (LTAR) and a Label-Context Aware Representation (LCAR) for enhanced semantic representations. Finally, the Trigger Classification module makes structured predictions, where each label is predicted with respect to its neighbours. We conduct experiments on three benchmark BioNLP datasets, namely MLEE, GE09, and GE11, to evaluate our proposed BioLSL model. Results show that BioLSL has achieved state-of-the-art performance, outperforming the baseline models. CONCLUSIONS: The proposed BioLSL model demonstrates good performance for biomedical event trigger detection without using any external resources. This suggests that label representation learning and context-aware enhancement are promising directions for improving the task. The key enhancement is that BioLSL effectively learns to construct semantic linkages between the event mentions and type labels, which provide the latent information of label-trigger and label-context relationships in biomedical texts. Moreover, additional experiments on BioLSL show that it performs exceptionally well with limited training data under the data-scarce scenarios.

In natural language processing, fact verification is a very challenging task, which requires retrieving multiple evidence sentences from a reliable corpus to verify the authenticity of a claim. Although most of the current deep learning methods use the attention mechanism for fact verification, they have not considered imposing attentional constraints on important related words in the claim and evidence sentences, resulting in inaccurate attention for some irrelevant words. In this paper, we propose a syntactic evidence network (SENet) model which incorporates entity keywords, syntactic information and sentence attention for fact verification. The SENet model extracts entity keywords from claim and evidence sentences, and uses a pre-trained syntactic dependency parser to extract the corresponding syntactic sentence structures and incorporates the extracted syntactic information into the attention mechanism for language-driven word representation. In addition, the sentence attention mechanism is applied to obtain a richer semantic representation. We have conducted experiments on the FEVER and UKP Snopes datasets for performance evaluation. Our SENet model has achieved 78.69% in Label Accuracy and 75.63% in FEVER Score on the FEVER dataset. In addition, our SENet model also has achieved 65.0% in precision and 61.2% in macro F1 on the UKP Snopes dataset. The experimental results have shown that our proposed SENet model has outperformed the baseline models and achieved the state-of-the-art performance for fact verification.

In task-oriented dialogue systems, Dialogue State Tracking (DST) aims to extract users' intentions from the dialogue history. Currently, most existing approaches suffer from error propagation and are unable to dynamically select relevant information when utilizing previous dialogue states. Moreover, the relations between the updates of different slots provide vital clues for DST. However, the existing approaches rely only on predefined graphs to indirectly capture the relations. In this paper, we propose a Dialogue State Distillation Network (DSDN) to utilize relevant information of previous dialogue states and migrate the gap of utilization between training and testing. Thus, it can dynamically exploit previous dialogue states and avoid introducing error propagation simultaneously. Further, we propose an inter-slot contrastive learning loss to effectively capture the slot co-update relations from dialogue context. Experiments are conducted on the widely used MultiWOZ 2.0 and MultiWOZ 2.1 datasets. The experimental results show that our proposed model achieves the state-of-the-art performance for DST.

Multilingual language models (MLLMs) have demonstrated remarkable success in various cross-lingual downstream tasks, facilitating the transfer of knowledge across numerous languages, whereas this transfer is not universally effective. Our study reveals that while existing MLLMs like mBERT can capturephrase-level alignments across the language families, they struggle to effectively capturesentence-level andparagraph-level alignments. To address this limitation, we propose GranCATs, Granularity-specific Contrastive AdapTers. We collect a new dataset that observes each sample at three distinct levels of granularity and employ contrastive learning as a pre-training task to train GranCATs on this dataset. Our objective is to enhance MLLMs' adaptation to a broader range of cross-lingual tasks by equipping them with improved capabilities to capture global information at different levels of granularity. Extensive experiments show that MLLMs with GranCATs yield significant performance advancements across various language tasks with different text granularities, including entity alignment, relation extraction, sentence classification and retrieval, and question-answering. These results validate the effectiveness of our proposed GranCATs in enhancing cross-lingual alignments across various text granularities and effectively transferring this knowledge to downstream tasks.

Fact verification is a challenging task that requires the retrieval of multiple pieces of evidence from a reliable corpus for verifying the truthfulness of a claim. Although the current methods have achieved satisfactory performance, they still suffer from one or more of the following three problems: (1) unable to extract sufficient contextual information from the evidence sentences; (2) containing redundant evidence information and (3) incapable of capturing the interaction between claim and evidence. To tackle the problems, we propose an evidence fusion network called EvidenceNet. The proposed EvidenceNet model captures global contextual information from various levels of evidence information for deep understanding. Moreover, a gating mechanism is designed to filter out redundant information in evidence. In addition, a symmetrical interaction attention mechanism is also proposed for identifying the interaction between claim and evidence. We conduct extensive experiments based on the FEVER dataset. The experimental results have shown that the proposed EvidenceNet model outperforms the current fact verification methods and achieves the state-of-the-art performance.

Fei Li, Zheng Wang, Siu Cheung Hui, Lejian Liao, Dandan Song, Jing Xu, Guoxiu He, Meihuizi Jia. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Named Entity Recognition (NER) is a fundamental problem in Natural Language Processing and has received much research attention. Although the current neural-based NER approaches have achieved the state-of-the-art performance, they still suffer from one or more of the following three problems in their architectures: (1) boundary tag sparsity, (2) lacking of global decoding information; and (3) boundary error propagation. In this paper, we propose a novel Boundary-aware Bidirectional Neural Networks (Ba-BNN) model to tackle these problems for neural-based NER. The proposed Ba-BNN model is constructed based on the structure of pointer networks for tackling the first problem on boundary tag sparsity. Moreover, we also use a boundary-aware binary classifier to capture the global decoding information as input to the decoders. In the Ba-BNN model, we propose to use two decoders to process the information in two different directions (i.e., from left-to-right and right-to-left). The final hidden states of the left-to-right decoder are obtained by incorporating the hidden states of the right-to-left decoder in the decoding process. In addition, a boundary retraining strategy is also proposed to help reduce boundary error propagation caused by the pointer networks in boundary detection and entity classification. We have conducted extensive experiments based on three NER benchmark datasets. The performance results have shown that the proposed Ba-BNN model has outperformed the current state-of-the-art models.

Factorization Machines (FMs) are a class of popular algorithms that have been widely adopted for collaborative filtering and recommendation tasks. FMs are characterized by its usage of the inner product of factorized parameters to model pairwise feature interactions, making it highly expressive and powerful. This paper proposes Holographic Factorization Machines (HFM), a new novel method of enhancing the representation capability of FMs without increasing its parameter size. Our approach replaces the inner product in FMs with holographic reduced representations (HRRs), which are theoretically motivated by associative retrieval and compressed outer products. Empirically, we found that this leads to consistent improvements over vanilla FMs by up to 4% improvement in terms of mean squared error, with improvements larger at smaller parameterization. Additionally, we propose a neural adaptation of HFM which enhances its capability to handle nonlinear structures. We conduct extensive experiments on nine publicly available datasets for collaborative filtering with explicit feedback. HFM achieves state-of-theart performance on all nine, outperforming strong competitors such as Attentional Factorization Machines (AFM) and Neural Matrix Factorization (NeuMF).

Yi Tay, Shuohang Wang, Anh Tuan Luu, Jie Fu, Minh C. Phan, Xingdi Yuan, Jinfeng Rao, Siu Cheung Hui, Aston Zhang. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

Many state-of-the-art neural models for NLP are heavily parameterized and thus memory inefficient.This paper proposes a series of lightweight and memory efficient neural architectures for a potpourri of natural language processing (NLP) tasks.To this end, our models exploit computation using Quaternion algebra and hypercomplex spaces, enabling not only expressive inter-component interactions but also significantly (75%) reduced parameter size due to lesser degrees of freedom in the Hamilton product.We propose Quaternion variants of models, giving rise to new architectures such as the Quaternion attention Model and Quaternion Transformer.Extensive experiments on a battery of NLP tasks demonstrates the utility of proposed Quaternion-inspired models, enabling up to 75% reduction in parameter size without significant loss in performance.

Local differential privacy (LDP) is a recently proposed privacy standard for collecting and analyzing data, which has been used, e.g., in the Chrome browser, iOS and macOS. In LDP, each user perturbs her information locally, and only sends the randomized version to an aggregator who performs analyses, which protects both the users and the aggregator against private information leaks. Although LDP has attracted much research attention in recent years, the majority of existing work focuses on applying LDP to complex data and/or analysis tasks. In this paper, we point out that the fundamental problem of collecting multidimensional data under LDP has not been addressed sufficiently, and there remains much room for improvement even for basic tasks such as computing the mean value over a single numeric attribute under LDP. Motivated by this, we first propose novel LDP mechanisms for collecting a numeric attribute, whose accuracy is at least no worse (and usually better) than existing solutions in terms of worst-case noise variance. Then, we extend these mechanisms to multidimensional data that can contain both numeric and categorical attributes, where our mechanisms always outperform existing solutions regarding worst-case noise variance. As a case study, we apply our solutions to build an LDP-compliant stochastic gradient descent algorithm (SGD), which powers many important machine learning tasks. Experiments using real datasets confirm the effectiveness of our methods, and their advantages over existing solutions.

Attentional models are distinctly characterized by their ability to learn relative importance, i.e., assigning a different weight to input values. This paper proposes a new quasi-attention that is compositional in nature, i.e., learning whether to \textit{add}, \textit{subtract} or \textit{nullify} a certain vector when learning representations. This is strongly contrasted with vanilla attention, which simply re-weights input tokens. Our proposed \textit{Compositional De-Attention} (CoDA) is fundamentally built upon the intuition of both similarity and dissimilarity (negative affinity) when computing affinity scores, benefiting from a greater extent of expressiveness. We evaluate CoDA on six NLP tasks, i.e. open domain question answering, retrieval/ranking, natural language inference, machine translation, sentiment analysis and text2code generation. We obtain promising experimental results, achieving state-of-the-art performance on several tasks/datasets.

Deep learning has demonstrated tremendous potential for Automatic Text Scoring (ATS) tasks. In this paper, we describe a new neural architecture that enhances vanilla neural network models with auxiliary neural coherence features. Our new method proposes a new SkipFlow mechanism that models relationships between snapshots of the hidden representations of a long short-term memory (LSTM) network as it reads. Subsequently, the semantic relationships between multiple snapshots are used as auxiliary features for prediction. This has two main benefits. Firstly, essays are typically long sequences and therefore the memorization capability of the LSTM network may be insufficient. Implicit access to multiple snapshots can alleviate this problem by acting as a protection against vanishing gradients. The parameters of the SkipFlow mechanism also acts as an auxiliary memory. Secondly, modeling relationships between multiple positions allows our model to learn features that represent and approximate textual coherence. In our model, we call this neural coherence features. Overall, we present a unified deep learning architecture that generates neural coherence features as it reads in an end-to-end fashion. Our approach demonstrates state-of-the-art performance on the benchmark ASAP dataset, outperforming not only feature engineering baselines but also other deep learning models.

Temporal gates play a significant role in modern recurrent-based neural encoders, enabling fine-grained control over recursive compositional operations over time. In recurrent models such as the long short-term memory (LSTM), temporal gates control the amount of information retained or discarded over time, not only playing an important role in influencing the learned representations but also serving as a protection against vanishing gradients. This paper explores the idea of learning temporal gates for sequence pairs (question and answer), jointly influencing the learned representations in a pairwise manner. In our approach, temporal gates are learned via 1D convolutional layers and then subsequently cross applied across question and answer for joint learning. Empirically, we show that this conceptually simple sharing of temporal gates can lead to competitive performance across multiple benchmarks. Intuitively, what our network achieves can be interpreted as learning representations of question and answer pairs that are aware of what each other is remembering or forgetting, i.e., pairwise temporal gating. Via extensive experiments, we show that our proposed model achieves state-of-the-art performance on two community-based QA datasets and competitive performance on one factoid-based QA dataset.

Aspect-based sentiment analysis (ABSA) tries to predict the polarity of a given document with respect to a given aspect entity. While neural network architectures have been successful in predicting the overall polarity of sentences, aspect-specific sentiment analysis still remains as an open problem. In this paper, we propose a novel method for integrating aspect information into the neural model. More specifically, we incorporate aspect information into the neural model by modeling word-aspect relationships. Our novel model, Aspect Fusion LSTM (AF-LSTM) learns to attend based on associative relationships between sentence words and aspect which allows our model to adaptively focus on the correct words given an aspect term. This ameliorates the flaws of other state-of-the-art models that utilize naive concatenations to model word-aspect similarity. Instead, our model adopts circular convolution and circular correlation to model the similarity between aspect and words and elegantly incorporates this within a differentiable neural attention framework. Finally, our model is end-to-end differentiable and highly related to convolution-correlation (holographic like) memories. Our proposed neural model achieves state-of-the-art performance on benchmark datasets, outperforming ATAE-LSTM by 4%-5% on average across multiple datasets.

Sarcasm is a sophisticated speech act which commonly manifests on social communities such as Twitter and Reddit.The prevalence of sarcasm on the social web is highly disruptive to opinion mining systems due to not only its tendency of polarity flipping but also usage of figurative language.Sarcasm commonly manifests with a contrastive theme either between positive-negative sentiments or between literal-figurative scenarios.In this paper, we revisit the notion of modeling contrast in order to reason with sarcasm.More specifically, we propose an attention-based neural model that looks inbetween instead of across, enabling it to explicitly model contrast and incongruity.We conduct extensive experiments on six benchmark datasets from Twitter, Reddit and the Internet Argument Corpus.Our proposed model not only achieves stateof-the-art performance on all datasets but also enjoys improved interpretability.

Data privacy is a major concern in modern society. In this work, we propose two solutions to the privacy-preserving problem of regression models on medical data. We focus on flexible parametric models which are powerful alternatives to the well-known Cox regression model. For the first approach, we propose a sampling mechanism which guarantees differential privacy for flexible parametric survival models. We first transform the likelihood function of the models to guarantee that likelihood values are bounded. We then use a Hamiltonian Monte-Carlo sampler to sample a random parameter vector from the posterior distribution. As a result, this random vector satisfies the requirement for differential privacy. For the second approach, as predictions with high accuracy and high confidence are very important for medical applications, we propose a mechanism which protects privacy by randomly perturbing the posterior distribution. We can then use the sampler to draw multiple random samples of the perturbed posterior to estimate the credible intervals of the parameters. The proposed mechanism does not guarantee differential privacy for the perturbed posterior. However, it allows controlling the contribution of each individual data record to the posterior. In the worst case scenario, when all data records are revealed except the target data record, the random noise added to the posterior would make it extremely difficult to obtain the target data record. The experiments conducted on two real datasets show that our proposed approaches outperform state-of-the-art methods in predicting the survival rate of individuals.

This paper presents a new deep learning architecture for Natural Language Inference (NLI).Firstly, we introduce a new architecture where alignment pairs are compared, compressed and then propagated to upper layers for enhanced representation learning.Secondly, we adopt factorization layers for efficient and expressive compression of alignment vectors into scalar features, which are then used to augment the base word representations.The design of our approach is aimed to be conceptually simple, compact and yet powerful.We conduct experiments on three popular benchmarks, SNLI, MultiNLI and SciTail, achieving competitive performance on all.A lightweight parameterization of our model also enjoys a ≈ 3 times reduction in parameter size compared to the existing state-of-the-art models, e.g., ESIM and DIIN, while maintaining competitive performance.Additionally, visual analysis shows that our propagated features are highly interpretable.

Sequence encoders are crucial components in many neural architectures for learning to read and comprehend.This paper presents a new compositional encoder for reading comprehension (RC).Our proposed encoder is not only aimed at being fast but also expressive.Specifically, the key novelty behind our encoder is that it explicitly models across multiple granularities using a new dilated composition mechanism.In our approach, gating functions are learned by modeling relationships and reasoning over multi-granular sequence information, enabling compositional learning that is aware of both long and short term information.We conduct experiments on three RC datasets, showing that our proposed encoder demonstrates very promising results both as a standalone encoder as well as a complementary building block.Empirical results show that simple Bi-Attentive architectures augmented with our proposed encoder not only achieves state-of-the-art / highly competitive results but is also considerably faster than other published works.

Learning a matching function between two text sequences is a long standing problem in NLP research.This task enables many potential applications such as question answering and paraphrase identification.This paper proposes Co-Stack Residual Affinity Networks (CSRAN), a new and universal neural architecture for this problem.CSRAN is a deep architecture, involving stacked (multi-layered) recurrent encoders.Stacked/Deep architectures are traditionally difficult to train, due to the inherent weaknesses such as difficulty with feature propagation and vanishing gradients.CSRAN incorporates two novel components to take advantage of the stacked architecture.Firstly, it introduces a new bidirectional alignment mechanism that learns affinity weights by fusing sequence pairs across stacked hierarchies.Secondly, it leverages a multi-level attention refinement component between stacked recurrent layers.The key intuition is that, by leveraging information across all network hierarchies, we can not only improve gradient flow but also improve overall performance.We conduct extensive experiments on six well-studied text sequence matching datasets, achieving state-of-the-art performance on all.

This paper proposes a new neural architecture that exploits readily available sentiment lexicon resources.The key idea is that that incorporating a word-level prior can aid in the representation learning process, eventually improving model performance.To this end, our model employs two distinctly unique components, i.e., (1) we introduce a lexicon-driven contextual attention mechanism to imbue lexicon words with long-range contextual information and (2), we introduce a contrastive co-attention mechanism that models contrasting polarities between all positive and negative words in a sentence.Via extensive experiments, we show that our approach outperforms many other neural baselines on sentiment classification tasks on multiple benchmark datasets.

Co-Attentions are highly effective attention mechanisms for text matching applications. Co-Attention enables the learning of pairwise attentions, i.e., learning to attend based on computing word-level affinity scores between two documents. However, text matching problems can exist in either symmetrical or asymmetrical domains. For example, paraphrase identification is a symmetrical task while question-answer matching and entailment classification are considered asymmetrical domains. In this paper, we argue that Co-Attention models in asymmetrical domains require different treatment as opposed to symmetrical domains, i.e., a concept of word-level directionality should be incorporated while learning word-level similarity scores. Hence, the standard inner product in real space commonly adopted in co-attention is not suitable. This paper leverages attractive properties of the complex vector space and proposes a co-attention mechanism based on the complex-valued inner product (Hermitian products). Unlike the real dot product, the dot product in complex space is asymmetric because the first item is conjugated. Aside from modeling and encoding directionality, our proposed approach also enhances the representation learning process. Extensive experiments on five text matching benchmark datasets demonstrate the effectiveness of our approach.

Many recent state-of-the-art recommender systems such as D-ATT, TransNet and DeepCoNN exploit reviews for representation learning. This paper proposes a new neural architecture for recommendation with reviews. Our model operates on a multi-hierarchical paradigm and is based on the intuition that not all reviews are created equal, i.e., only a selected few are important. The importance, however, should be dynamically inferred depending on the current target. To this end, we propose a review-by-review pointer-based learning scheme that extracts important reviews from user and item reviews and subsequently matches them in a word-by-word fashion. This enables not only the most informative reviews to be utilized for prediction but also a deeper word-level interaction. Our pointer-based method operates with a gumbel-softmax based pointer mechanism that enables the incorporation of discrete vectors within differentiable neural architectures. Our pointer mechanism is co-attentive in nature, learning pointers which are co-dependent on user-item relationships. Finally, we propose a multi-pointer learning scheme that learns to combine multiple views of user-item interactions. We demonstrate the effectiveness of our proposed model via extensive experiments on 24 benchmark datasets from Amazon and Yelp. Empirical results show that our approach significantly outperforms existing state-of-the-art models, with up to 19% and 71% relative improvement when compared to TransNet and DeepCoNN respectively. We study the behavior of our multi-pointer learning mechanism, shedding light on 'evidence aggregation' patterns in review-based recommender systems.

Attention is typically used to select informative sub-phrases that are used for prediction. This paper investigates the novel use of attention as a form of feature augmentation, i.e, casted attention. We propose Multi-Cast Attention Networks (MCAN), a new attention mechanism and general model architecture for a potpourri of ranking tasks in the conversational modeling and question answering domains. Our approach performs a series of soft attention operations, each time casting a scalar feature upon the inner word embeddings. The key idea is to provide a real-valued hint (feature) to a subsequent encoder layer and is targeted at improving the representation learning process. There are several advantages to this design, e.g., it allows an arbitrary number of attention mechanisms to be casted, allowing for multiple attention types (e.g., co-attention, intra-attention) and attention variants (e.g., alignment-pooling, max-pooling, mean-pooling) to be executed simultaneously. This not only eliminates the costly need to tune the nature of the co-attention layer, but also provides greater extents of explainability to practitioners. Via extensive experiments on four well-known benchmark datasets, we show that MCAN achieves state-of-the-art performance. On the Ubuntu Dialogue Corpus, MCAN outperforms existing state-of-the-art models by 9%. MCAN also achieves the best performing score to date on the well-studied TrecQA dataset.

Recurrent neural networks (RNNs) such as long short-term memory and gated recurrent units are pivotal building blocks across a broad spectrum of sequence modeling problems. This paper proposes a recurrently controlled recurrent network (RCRN) for expressive and powerful sequence encoding. More concretely, the key idea behind our approach is to learn the recurrent gating functions using recurrent networks. Our architecture is split into two components - a controller cell and a listener cell whereby the recurrent controller actively influences the compositionality of the listener cell. We conduct extensive experiments on a myriad of tasks in the NLP domain such as sentiment analysis (SST, IMDb, Amazon reviews, etc.), question classification (TREC), entailment classification (SNLI, SciTail), answer selection (WikiQA, TrecQA) and reading comprehension (NarrativeQA). Across all 26 datasets, our results demonstrate that RCRN not only consistently outperforms BiLSTMs but also stacked BiLSTMs, suggesting that our controller architecture might be a suitable replacement for the widely adopted stacked architecture. Additionally, RCRN achieves state-of-the-art results on several well-established datasets.

We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors. To this end, we propose novel Bidirectional Attention Connectors (BAC) for efficiently forging connections throughout the network. We conduct extensive experiments on four challenging RC benchmarks. Our proposed approach achieves state-of-the-art results on all four, outperforming existing baselines by up to 2.6% to 14.2% in absolute F1 score.

The dominant neural architectures in question answer retrieval are based on recurrent or convolutional encoders configured with complex word matching layers. Given that recent architectural innovations are mostly new word interaction layers or attention-based matching mechanisms, it seems to be a well-established fact that these components are mandatory for good performance. Unfortunately, the memory and computation cost incurred by these complex mechanisms are undesirable for practical applications. As such, this paper tackles the question of whether it is possible to achieve competitive performance with simple neural architectures. We propose a simple but novel deep learning architecture for fast and efficient question-answer ranking and retrieval. More specifically, our proposed model, HyperQA, is a parameter efficient neural network that outperforms other parameter intensive models such as Attentive Pooling BiLSTMs and Multi-Perspective CNNs on multiple QA benchmarks. The novelty behind HyperQA is a pairwise ranking objective that models the relationship between question and answer embeddings in Hyperbolic space instead of Euclidean space. This empowers our model with a self-organizing ability and enables automatic discovery of latent hierarchies while learning embeddings of questions and answers. Our model requires no feature engineering, no similarity matrix matching, no complicated attention mechanisms nor over-parameterized layers and yet outperforms and remains competitive to many models that have these functionalities on multiple benchmarks.

This paper proposes a new neural architecture for collaborative ranking with implicit feedback. Our model, LRML (Latent Relational Metric Learning) is a novel metric learning approach for recommendation. More specifically, instead of simple push-pull mechanisms between user and item pairs, we propose to learn latent relations that describe each user item interaction. This helps to alleviate the potential geometric inflexibility of existing metric learning approaches. This enables not only better performance but also a greater extent of modeling capability, allowing our model to scale to a larger number of interactions. In order to do so, we employ a augmented memory module and learn to attend over these memory blocks to construct latent relations. The memory-based attention module is controlled by the user-item interaction, making the learned relation vector specific to each user-item pair. Hence, this can be interpreted as learning an exclusive and optimal relational translation for each user-item interaction. The proposed architecture demonstrates the state-of-the-art performance across multiple recommendation benchmarks. LRML outperforms other metric learning models by 6%-7.5% in terms of [email protected] and [email protected] on large datasets such as Netflix and MovieLens20M. Moreover, qualitative studies also demonstrate evidence that our proposed model is able to infer and encode explicit sentiment, temporal and attribute information despite being only trained on implicit feedback. As such, this ascertains the ability of LRML to uncover hidden relational structure within implicit datasets.

Abstract Learning English grammar is a very challenging task for many students especially for nonnative English speakers. To learn English well, it is important to understand the concepts of the English grammar with lots of practise on exercise questions. Previous recommendation systems for learning English mainly focused on recommending reading materials and vocabulary. Different from reading material and vocabulary recommendations, grammar question recommendation should recommend questions that have similar grammatical structure and usage to the question of interest. The content similarity calculation methods used in existing recommendation methods cannot represent the similarity between grammar questions effectively. In this paper, we propose a content‐based approach for personalized grammar question recommendation, which recommends similar grammatical structure and usage questions for further practising. Specifically, we propose a novel structure namedparse‐key treeto capture the grammatical structure and usage of grammar questions. We then propose 3 measures to compute the similarity between the question query and database questions for grammar question recommendation. Additionally, we incorporated the proposed recommendation method into a Web‐based English grammar learning system and presented its performance evaluation in this paper. The experimental results have shown that the proposed approach outperforms other classical and state‐of‐the‐art methods in recommending relevant grammar questions.

With the popularity of online educational platforms, English learners can learn and practice no matter where they are and what they do. English grammar is one of the important components in learning English. To learn English grammar effectively, it requires students to practice questions containing focused grammar knowledge. In this paper, we study a novel problem of retrieving English grammar questions with similar grammatical focus. Since the grammatical focus similarity is different from textual similarity or sentence syntactic similarity, existing approaches cannot be applied directly to our problem. To address this problem, we propose a syntactic based approach for English grammar question retrieval which can retrieve related grammar questions with similar grammatical focus effectively. In the proposed syntactic based approach, we first propose a new syntactic tree, namely parse-key tree, to capture English grammar questions’ grammatical focus. Next, we propose two kernel functions , namely relaxed tree kernel and part-of-speech order kernel, to compute the similarity between two parse-key trees of the query and grammar questions in the collection. Then, the retrieved grammar questions are ranked according to the similarity between the parse-key trees. In addition, if a query is submitted together with answer choices, conceptual similarity and textual similarity are also incorporated to further improve the retrieval accuracy . The performance results have shown that our proposed approach outperforms the state-of-the-art methods based on statistical analysis and syntactic analysis.

Knowledge graphs play a significant role in many intelligent systems such as semantic search and recommendation systems. Recent works in this area of knowledge graph embeddings such as TransE, TransH and TransR have shown extremely competitive and promising results in relational learning. In this paper, we propose a novel extension of the translational embedding model to solve three main problems of the current models. Firstly, translational models are highly sensitive to hyperparameters such as margin and learning rate. Secondly, the translation principle only allows one spot in vector space for each golden triplet. Thus, congestion of entities and relations in vector space may reduce precision. Lastly, the current models are not able to handle dynamic data especially the introduction of new unseen entities/relations or removal of triplets. In this paper, we propose Parallel Universe TransE (puTransE), an adaptable and robust adaptation of the translational model. Our approach non-parametrically estimates the energy score of a triplet from multiple embedding spaces of structurally and semantically aware triplet selection. Our proposed approach is simple, robust and parallelizable. Our experimental results show that our proposed approach outperforms TransE and many other embedding methods for link prediction on knowledge graphs on both public benchmark dataset and a real world dynamic dataset.

In survival analysis, regression models are used to understand the effects of explanatory variables (e.g., age, sex, weight, etc.) to the survival probability. However, for sensitive survival data such as medical data, there are serious concerns about the privacy of individuals in the data set when medical data is used to fit the regression models. The closest work addressing such privacy concerns is the work on Cox regression which linearly projects the original data to a lower dimensional space. However, the weakness of this approach is that there is no formal privacy guarantee for such projection. In this work, we aim to propose solutions for the regression problem in survival analysis with the protection of differential privacy which is a golden standard of privacy protection in data privacy research. To this end, we extend the Output Perturbation and Objective Perturbation approaches which are originally proposed to protect differential privacy for the Empirical Risk Minimization (ERM) problems. In addition, we also propose a novel sampling approach based on the Markov Chain Monte Carlo (MCMC) method to practically guarantee differential privacy with better accuracy. We show that our proposed approaches achieve good accuracy as compared to the non-private results while guaranteeing differential privacy for individuals in the private data set.

This paper proposes Dyadic Memory Networks (DyMemNN), a novel extension of end-to-end memory networks (memNN) for aspect-based sentiment analysis (ABSA). Originally designed for question answering tasks, memNN operates via a memory selection operation in which relevant memory pieces are adaptively selected based on the input query. In the problem of ABSA, this is analogous to aspects and documents in which the relationship between each word in the document is compared with the aspect vector. In the standard memory networks, simple dot products or feed forward neural networks are used to model the relationship between aspect and words which lacks representation learning capability. As such, our dyadic memory networks ameliorates this weakness by enabling rich dyadic interactions between aspect and word embeddings by integrating either parameterized neural tensor compositions or holographic compositions into the memory selection operation. To this end, we propose two variations of our dyadic memory networks, namely the Tensor DyMemNN and Holo DyMemNN. Overall, our two models are end-to-end neural architectures that enable rich dyadic interaction between aspect and document which intuitively leads to better performance. Via extensive experiments, we show that our proposed models achieve the state-of-the-art performance and outperform many neural architectures across six benchmark datasets.

Many popular knowledge graphs such as Freebase, YAGO or DBPedia maintain a list of non-discrete attributes for each entity. Intuitively, these attributes such as height, price or population count are able to richly characterize entities in knowledge graphs. This additional source of information may help to alleviate the inherent sparsity and incompleteness problem that are prevalent in knowledge graphs. Unfortunately, many state-of-the-art relational learning models ignore this information due to the challenging nature of dealing with non-discrete data types in the inherently binary-natured knowledge graphs. In this paper, we propose a novel multi-task neural network approach for both encoding and prediction of non-discrete attribute information in a relational setting. Specifically, we train a neural network for triplet prediction along with a separate network for attribute value regression. Via multi-task learning, we are able to learn representations of entities, relations and attributes that encode information about both tasks. Moreover, such attributes are not only central to many predictive tasks as an information source but also as a prediction target. Therefore, models that are able to encode, incorporate and predict such information in a relational learning context are highly attractive as well. We show that our approach outperforms many state-of-the-art methods for the tasks of relational triplet classification and attribute value prediction.

We describe a new deep learning architecture for learning to rank question answer pairs. Our approach extends the long short-term memory (LSTM) network with holographic composition to model the relationship between question and answer representations. As opposed to the neural tensor layer that has been adopted recently, the holographic composition provides the benefits of scalable and rich representational learning approach without incurring huge parameter costs. Overall, we present Holographic Dual LSTM (HD-LSTM), a unified architecture for both deep sentence modeling and semantic matching. Essentially, our model is trained end-to-end whereby the parameters of the LSTM are optimized in a way that best explains the correlation between question and answer representations. In addition, our proposed deep learning architecture requires no extensive feature engineering. Via extensive experiments, we show that HD-LSTM outperforms many other neural architectures on two popular benchmark QA datasets. Empirical studies confirm the effectiveness of holographic composition over the neural tensor layer.

Link prediction on knowledge graphs is useful in numerous application areas such as semantic search, question answering, entity disambiguation, enterprise decision support, recommender systems and so on. While many of these applications require a reasonably quick response and may operate on data that is constantly changing, existing methods often lack speed and adaptability to cope with these requirements. This is aggravated by the fact that knowledge graphs are often extremely large and may easily contain millions of entities rendering many of these methods impractical. In this paper, we address the weaknesses of current methods by proposing Random Semantic Tensor Ensemble (RSTE), a scalable ensemble-enabled framework based on tensor factorization. Our proposed approach samples a knowledge graph tensor in its graph representation and performs link prediction via ensembles of tensor factorization. Our experiments on both publicly available datasets and real world enterprise/sales knowledge bases have shown that our approach is not only highly scalable, parallelizable and memory efficient, but also able to increase the prediction accuracy significantly across all datasets.

Parallel test paper generation is a biobjective distributed resource optimization problem, which aims to generate multiple similarly optimal test papers automatically according to multiple user-specified assessment criteria. Generating high-quality parallel test papers is challenging due to its NP-hardness in both of the collective objective functions. In this paper, we propose a submodular memetic approximation algorithm for solving this problem. The proposed algorithm is an adaptive memetic algorithm (MA), which exploits the submodular property of the collective objective functions to design greedy-based approximation algorithms for enhancing steps of the multiobjective MA. Synergizing the intensification of submodular local search mechanism with the diversification of the population-based submodular crossover operator, our algorithm can jointly optimize the total quality maximization objective and the fairness quality maximization objective. Our MA can achieve provable near-optimal solutions in a huge search space of large datasets in efficient polynomial runtime. Performance results on various datasets have shown that our algorithm has drastically outperformed the current techniques in terms of paper quality and runtime efficiency.

Taxonomic relation identification aims to recognize the 'is-a' relation between two terms.Previous works on identifying taxonomic relations are mostly based on statistical and linguistic approaches, but the accuracy of these approaches is far from satisfactory.In this paper, we propose a novel supervised learning approach for identifying taxonomic relations using term embeddings.For this purpose, we first design a dynamic weighting neural network to learn term embeddings based on not only the hypernym and hyponym terms, but also the contextual information between them.We then apply such embeddings as features to identify taxonomic relations using a supervised method.The experimental results show that our proposed approach significantly outperforms other state-of-the-art methods by 9% to 13% in terms of accuracy for both general and specific domain datasets.Recently, Yu et al. (2015) proposed a super-

Taxonomies play an important role in many applications by organizing domain knowledge into a hierarchy of ‘ is-a’ relations between terms. Previous work on automatic construction of taxonomies from text documents either ignored temporal information or used fixed time periods to discretize the time series of documents. In this paper, we propose a time-aware method to automatically construct and effectively maintain a taxonomy from a given series of documents preclustered for a domain of interest. The method extracts temporal information from the documents and uses a timestamp contribution function to score the temporal relevance of the evidence from source texts when identifying the taxonomic relations for constructing the taxonomy. Experimental results show that our proposed method outperforms the state-of-the-art methods by increasing F-measure up to 7%–20%. Furthermore, the proposed method can incrementally update the taxonomy by adding fresh relations from new data and removing outdated relations using an information decay function. It thus avoids rebuilding the whole taxonomy from scratch for every update and keeps the taxonomy effectively up-to-date in order to track the latest information trends in the rapidly evolving domain.

Recognizing handwritten chemical structural formulas is challenging due to the spatial complexity involved, especially for chemical skeletal structural formulas. In chemical skeletal structural formulas, the Carbon atoms are omitted in the drawing of chemical connected bonds, in which the single bonds are connected with hidden Carbon atoms as the connection points. To recognize chemical skeletal structural formulas, it is important to recognize the connected bond strokes and differentiate them from other similar symbols. Moreover, even if the connected bond stroke is identified, it is also challenging to split the connected bond stroke into several single bonds and identify the locations of the hidden Carbon atoms, and its positional relationship in the overall chemical structure. In this paper, we propose a progressive approach for recognizing chemical skeletal structural formulas, in particular the connected bonds. The proposed approach recognizes a chemical formula progressively while the user draws the chemical skeletal structural formula.

With the growing popularity of pen-based and touch-based devices such as Apple's iPad and Samsung's Galaxy Tablet, handwriting has become an important input method. Although handwriting recognition for text contents and mathematical formulae are well-supported in these devices, recognizing handwritten chemical expressions is still very challenging due to its complex spatial structure. In this research, we focus on chemical symbol recognition which is essential for accurate handwritten chemical expression recognition. In particular, we propose an online hybrid Support Vector Machine - Elastic Matching (SVM-EM) approach for handwritten chemical symbol recognition. Based on the proposed chemical symbol recognition approach and an online structural analysis, we have implemented an online handwritten chemical expression recognition system on Apple's iOS platform. In this paper, we present our proposed SVM-EM approach for handwritten chemical symbol recognition and evaluate it with several users to verify its promising performance as a real application.

With the recent emergence of pen-based and touch based input devices such as Apple's iPad and Samsung's Galaxy Tablet, it has become more feasible now to input chemical formulas directly by handwriting, which is more natural and efficient than the traditional template-based input methods. In this paper, we propose an effective graph-based chemical structural analysis approach for online progressive handwritten chemical formula recognition. The proposed approach can progressively generate the recognition result after recognizing each symbol and users can make any corrections to the recognition result immediately. In addition, the proposed approach can recognize both cyclic and non-cyclic chemical structures. Recognizing cyclic structural formulas is challenging as bond orientations are very flexible and the relationships between symbols are much more complex than non-cyclic structural formulas. In this paper, the proposed chemical structural analysis approach and its promising performance results will be presented.

We propose a math-aware search engine that is capable of handling both textual keywords as well as mathematical expressions. Our math feature extraction and representation framework captures the semantics of math expressions via a Finite State Machine model. We adapt the passive aggressive online learning binary classifier as the ranking model. We benchmarked our approach against three classical information retrieval (IR) strategies on math documents crawled from Math Overflow, a well-known online math question answering system. Experimental results show that our proposed approach can perform better than other methods by more than 9%.

The relationships between consumer emotions and their buying behaviors have been well documented. Technology-savvy consumers often use the web to find information on products and services before they commit to buying. We propose a semantic web usage mining approach for discovering periodic web access patterns from annotated web usage logs which incorporates information on consumer emotions and behaviors through self-reporting and behavioral tracking. We use fuzzy logic to represent real-life temporal concepts (e.g., morning) and requested resource attributes (ontological domain concepts for the requested URLs) of periodic pattern-based web access activities. These fuzzy temporal and resource representations, which contain both behavioral and emotional cues, are incorporated into a Personal Web Usage Lattice that models the user's web access activities. From this, we generate a Personal Web Usage Ontology written in OWL, which enables semantic web applications such as personalized web resources recommendation. Finally, we demonstrate the effectiveness of our approach by presenting experimental results in the context of personalized web resources recommendation with varying degrees of emotional influence. Emotional influence has been found to contribute positively to adaptation in personalized recommendation.

In centroid-based classification, each class is represented by a prototype or centroid document vector that is formed by averaging all member vectors during the training phase. In the prediction phase, the label of a test document vector is assigned to that of its nearest class prototype. Recently there has been revived interest in reformulating the prototype/centroid to improve classification performance. In this paper, we study the theoretical properties of the recently proposed Class Feature Centroid (CFC) classifier by considering the rate of change of each prototype vector with respect to individual dimensions (terms). The implication of our theoretical finding is that CFC is inherently biased towards large (dominant majority) classes, which means it is destined to perform poorly for highly class-imbalanced data. Another practical concern about CFC lies in its overly-aggressive design in weeding out terms that appear in all classes. To overcome these CFC limitations while retaining its intrinsic and worthy design goals, we propose an improved and robust centroid-based classifier that uses precise term-class distribution properties instead of simple presence or absence of terms in classes. Specifically, terms are weighted based on the Kullback-Leibler divergence measure between pairs of class-conditional term probabilities, we call this the CFC-KL centroid classifier. We then generalized CFC-KL to handle multi-class data by summing pair wise class-conditioned word probability ratios. Our proposed approach has been evaluated on 5 datasets, on which it consistently outperforms CFC and the baseline Support Vector Machine classifier. We also devise a word cloud visualization approach to highlight the important class-specific words picked out by our CFC-KL, and visually compare it with other popular term weigthing approaches. Our encouraging results show that the centroid based generalized CFC-KL classifier is both robust and efficient to deal with real-world text classification.

Vector space text classification is commonly used in intelligence applications such as email and conversation analysis. In this paper we propose a supervised term weighting scheme called tƒ × KL (term frequency Kullback-Leibler), which weights each word proportionally to the ratio of its document frequency across the positive and negative class. We then generalize tƒ × KL to effectively deal with class imbalance, which is very common in real world intelligence analysis. The generalized tƒ × KL weights each word according to the ratio of the positive and negative class conditioned word probabilities instead of the raw document frequencies. Results on four classification datasets show tƒ × KL to perform consistently better than the baseline tƒ ×idƒ and 4 other supervised term weighting schemes, including the recently proposed tƒ × rƒ (term frequency relevance frequency). The generalized tƒ × KL was found to be extremely robust in dealing with highly skewed class distributions, beating the second runner-up by more than 20% on a dataset that has only 10% positive training examples. The generalized tƒ × KL is thus an effective and robust term weighting scheme that can significantly improve binary classification performance in sentiment analysis and intelligence applications.

Associative classification (AC), which is based on association rules, has shown great promise over many other classification techniques. To implement AC effectively, we need to tackle the problems on the very large search space of candidate rules during the rule discovery process and incorporate the discovered association rules into the classification process. This paper proposes a new approach that we call artificial immune system-associative classification (AIS-AC), which is based on AIS, for mining association rules effectively for classification. Instead of massively searching for all possible association rules, AIS-AC will only find a subset of association rules that are suitable for effective AC in an evolutionary manner. In this paper, we also evaluate the performance of the proposed AIS-AC approach for AC based on large datasets. The performance results have shown that the proposed approach is efficient in dealing with the problem on the complexity of the rule search space, and at the same time, good classification accuracy has been achieved. This is especially important for mining association rules from large datasets in which the search space of rules is huge.

Periodic Web personalization aims to recommend the most relevant resources to a user during a specific time period by analyzing the periodic access patterns of the user from Web usage logs. In this paper, we propose a novel Web usage mining approach for supporting effective periodic Web personalization. The proposed approach first constructs a user behavior model, called personal Web usage lattice, from Web usage logs using the fuzzy formal concept analysis technique. Based on the personal Web usage lattice, resources that the user is most probably interested in during a given period can be deduced efficiently. This approach enables the costly personalized resources preparation process to be done in advance rather than in real-time. The performance evaluation of the proposed periodic Web personalization approach is also given in the paper

Customer service support is an important operation for most multinational manufacturing companies. With the advancement of internet technologies, customer services nowadays are supported through web-based systems. More recently, rapid development of the semantic web and semantic web services has prompted us to develop a semantic help-desk for supporting customer services over the semantic web environment, which is presented in this paper. In particular, a fuzzy formal concept analysis (FCA)-based approach is developed for automatic generation of fuzzy machine service ontology that can deal with uncertain information. The proposed automatic fuzzy ontology generation technique consists of the following steps: fuzzy formal concept analysis, fuzzy conceptual clustering, and ontology generation. As such, the supporting machine services provided by the proposed system will potentially improve customer satisfaction in terms of reducing machine down time and increasing productivity. In this paper, an experiment has also been conducted for performance evaluation. The experimental result shows that the proposed approach has attained good performance in terms of both accuracy and efficiency when the queries are associated with appropriate membership values, and a suitable confident threshold is set.

Ontology is an effective conceptualism commonly used for the semantic Web. Fuzzy logic can be incorporated to ontology to represent uncertainty information. Typically, fuzzy ontology is generated from a predefined concept hierarchy. However, to construct a concept hierarchy for a certain domain can be a difficult and tedious task. To tackle this problem, this paper proposes the FOGA (fuzzy ontology generation framework) for automatic generation of fuzzy ontology on uncertainty information. The FOGA framework comprises the following components: fuzzy formal concept analysis, concept hierarchy generation, and fuzzy ontology generation. We also discuss approximating reasoning for incremental enrichment of the ontology with new upcoming data. Finally, a fuzzy-based technique for integrating other attributes of database to the ontology is proposed.

Discovering and understanding Web users' surfing behavior are essential for the development of successful Web monitoring and recommendation systems. In this paper, we propose a Web usage mining approach for the automatic discovery and visualization of temporal-based Web access behavior of individual users by mining client-side logs. The proposed approach is based on a Web usage lattice model which represents a hierarchy of Web access activities. To describe such Web access activities, we incorporate fuzzy logic to represent real life temporal concepts such as morning, afternoon and evening, and meaningful Web categories such as news, sports and chat. Based on the lattice, temporal and association behavior patterns can be extracted and visualized.

It is important to protect children and unsuspecting adults from the harmful effects of objectionable materials, such as pornography, violence, and hate messages, which are now prevalent on the World-Wide Web. This calls for effective tools for web content analysis and filtering of objectionable contents. Our study of existing web content filtering systems has identified a number of deficiencies in these systems. Using the analysis of pornographic web pages as a case study, we present an intelligent bilingual web page categorization engine that can determine if an English or Chinese language web page contains pornographic materials. We have implemented the categorization engine to perform offline web page analysis and near-instantaneous online filtering. Performance evaluation of our system has verified its effectiveness.

Currently, most peer-to-peer (P2P) systems are designed for file sharing by network participants. Simple meta-data search mechanism will be sufficient to support searching and retrieving shared files over P2P networks. However, to share document information such as news articles, scientific publications, company reports, etc., a content-based search mechanism is needed to provide efficient content-based retrieval. In this paper, we propose an intelligent P2P content-based document retrieval system known as iSearch-P2P. In iSearch-P2P, we have incorporated an intelligent technique based on the Fuzzy Adaptive Resonance Theory (Fuzzy ART) neural network to perform document clustering in order to support content-based publishing and retrieval over P2P networks. With intelligent content-based search, the iSearch-P2P system supports scalability and avoids indexing and query flooding problems of most existing P2P systems. In this paper, we describe the architecture, the publishing and retrieval processes, implementation and performance evaluation of the proposed iSearch-P2P system.

Finding expertise in a research area helps researchers to know whom are the experts working on the research area. This paper proposes a web mining approach for finding expertise in scientific research areas. In this approach, Indexing Agents search and download scientific publications from web sites that typically include academic web pages, then they extract citations and store them in a Web Citation Database. In addition, researcher information is also saved into the Researcher Database. Data mining techniques are applied to the Web Citation Database on citation keywords and authors to form document clusters and author clusters. The Multi-Clustering technique is proposed to mine the combined information of document clusters and author clusters for information on expertise in specified research areas.

The automatic recognition of facial expression presents a significant challenge to the pattern analysis and man-machine interaction research community. Recognition from a single static image is particularly a difficult task. In this paper, we present a methodology for facial expression recognition from a single static image using line-based caricatures. The recognition process is completely automatic. It also addresses the computational expensive problem and is thus suitable for real-time applications. The proposed approach uses structural and geometrical features of a user sketched expression model to match the line edge map (LEM) descriptor of an input face image. A disparity measure that is robust to expression variations is defined. The effectiveness of the proposed technique has been evaluated and promising results are obtained. This work has proven the proposed idea that facial expressions can be characterized and recognized by caricatures.

Face recognition is one of the most suitable biometric methods for supporting systems that require high security. The success of a face recognition system involves more than just the comparison algorithm. Face data acquisition is the first step towards developing face recognition systems. The accuracy of a face recognition system greatly depends on the face data acquired. This paper proposes the head-nodding method for face capturing using motion analysis. By analyzing the head-nodding motion of the user, the head-nodding method captures face images with consistent orientation to support face verification systems. In this paper, the design and implementation of the head-nodding method is presented. Preliminary experimental results are also presented.

The WWW has become one of the most important media for sharing information. Web information provides another emerging and important avenue and source of competitive intelligence (CI) for companies. CI is critical for companies to stay competitive in the marketplace. Apart from business users, there are other types of CI users such as technical users, casual users, news awareness users and others who would like to be kept informed on the latest development of their interested areas over the WWW. To discover web information, CI users need to constantly monitor certain web sites and web pages for related information. However, the dynamic nature of the web has made such monitoring task complicated and time-consuming. This paper proposes a web monitoring system, WebMon, to help users monitor specified web pages for latest changes and updates in information. Four monitoring functions including date monitoring, keywords monitoring, link monitoring and portion monitoring are supported by the system. The performance of these monitoring functions is also evaluated.

The Internet's increasing popularity and widespread acceptance have prompted its use as an alternative medium for fax, voice, and video communications to reduce cost by circumventing expensive international toll rates. Unified messaging integrates different media such as facsimile, text mail, voice mail, video mail, and pager-messages into a single mechanism for message submission, transportation, and retrieval. In this paper, a unified messaging environment over the Internet is proposed. This environment essentially comprises two gateways, namely, the front end gateway FEG and the back end gateway BEG to provide the necessary support for message dispatch and retrieval. Front end gateway is responsible for the message dispatch process including submission, preprocessing, packaging, and Internet delivery, while BEG uses a dual-contact technique for delivery of the unified message to the recipient over packet switched telephone network and the Web. The two gateways can be combined into a unified messaging gateway UMG system.

The study outlines a number of security requirements that are typical of a host of Web‐based applications using a case study of a real life online Web‐based customer support system. It subsequently proposes a security solution that employs a combination of Web server security measures and cryptographic techniques. The Web server security measures include the formulation and implementation of a policy for server physical security, configuration control, users’ access control and regular Web server log checks. Login passwords, in conjunction with public key cryptographic techniques and random nonces, are used to achieve user authentication, provide a safeguard against replay attacks, and prevent non‐repudiatory usage of system by users. These techniques, together with the use of session keys, will allow data integrity and confidentiality of the customer support system to be enforced. Furthermore, a number of security guidelines have been observed in the implementation of the relevant software to ensure further safety of the system.

With the information explosion from the Internet, there is a need to efficiently determine the relevance of information. This paper discusses an approach to information filtering using dynamic abstract generation techniques. Different abstract generation techniques such as the location method, indicative-phrases, keyword frequency, and title-keyword method are incorporated into a retrieval interface for on-line news articles. During news retrieval, abstract generation, an extract containing a set of verbatim sentences from the news article will be automatically produced. This will form an indicative abstract from which the prospective reader can then decide whether to read the full-length news article. In this way, a reader can filter out irrelevant news articles without having to review the entire article.

A classifier is proposed in this paper to extract structural information from business forms. The classifier is built upon existing techniques and takes advantage of the highly structured nature of forms, containing lines, boxes and text. Improvements are made to widen the scope of the classifier to handle some unexpected cases from real images. A syntactic representation is built from the detected features using their positions and lengths. The information recorded in this representation is independent of scale and displacement. A filled in form can then be compared to prerecorded blank forms to see which of these fits the best. Encouraging experimental results have been obtained.

An object-oriented approach for constructing an IC failure analysis system environment named Multi-ICFA is described. The MultiICFA enables the failure analyst to manage IC analysis information, which consists of video, graphical plot, film, image, and text description, through a user interface. In this paper, the characteristics of failure analysis in an IC device are described and a process of failure analysis, which is currently used by an IC manufacturer, is given. The architecture of the MultiICFA is outlined, which focuses on modelling and management of data. An object-oriented approach to design is adopted in the implementation of the system. The goal of the system is to enable the failure analyst to formulate event-driven queries through the user interface in order to display the test results of faulty IC devices.