Zhenguang Wang, Bo Li, Wenhui Tan, Peng Cao, Yang Wang, Jia Duan, Fei Wang, Osmar Zaiane. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Depression recognition is critical for early detection and treatment. Existing works focus on modeling coarse-grained multimodal representation to estimate the depression level. However, these approaches often overlook the inherent locality of depressive representation, resulting in weak and sparse depressive frames being overlooked. In addition, they neglect the inter modal correlations and intra-modal patterns of mood change, limiting the learning of multimodal complementary information. Therefore, we present a Locality-Aware Multimodal Depression (LAMD) recognition model. Specifically, LAMD contains three innovations: 1) Considering the sparsity of depressive features, we propose an Adaptive Temporal Attention (ATA) module to adaptively highlight keyframes with depressive features and suppress irrelevant frames. Additionally, we introduce Segment Information Sharing (SIS) strategy to overcome the limitation of inter-segment independence, enabling global awareness of depressive features within the whole segment. 2) We revisit the audio-video multimodal interaction from the perspectives of inter-modal correlation and intra-modal smoothness, introducing frame-level multimodal attention consistency constraints and smooth constraints. Furthermore, we propose a local cross attention to enhance the inter-modal interactions in adjacent time. 3) Extensive experiments on several datasets demonstrate that LAMD achieves superior performance, with up to 7.21 RMSE and 76.77% F1-score on the AVEC2014 and NJAD dataset, outperforming the prior art by a notable 0.22% and 1.88% margin, respectively. Moreover, visual analysis reveals that LAMD can adaptively perceive depressive keyframes and focus on fine-grained facial regions known for capturing subtle depressive expressions.

This paper explores a challenging yet realistic scenario: semi-supervised domain generalization (SSDG) that includes label scarcity and domain shift problems. We pinpoint that the limitations of previous SSDG methods lie in 1) neglecting the difference between domain shifts existing within a training dataset (intra-domain shift, IDS) and those occurring between training and testing datasets (cross-domain shift, CDS) and 2) overlooking the interplay between label scarcity and domain shifts, resulting in these methods merely stitching together semi-supervised learning (SSL) and domain generalization (DG) techniques. Considering these limitations, we propose a novel perspective to decompose SSDG into the combination of unsupervised domain adaptation (UDA) and DG problems. To this end, we design a causal augmentation and disentanglement framework (CausalAD) for semi-supervised domain generalized medical image segmentation. Concretely, CausalAD involves two collaborative processes: an augmentation process, which utilizes disentangled style factors to perform style augmentation for UDA, and a disentanglement process, which decouples domain-invariant (content) and domain-variant (noise and style) features for DG. Furthermore, we propose a proxy-based self-paced training strategy (ProSPT) to guide the training of CausalAD by gradually selecting unlabeled image pixels with high-quality pseudo labels in a self-paced training manner. Finally, we introduce a hierarchical structural causal model (HSCM) to explain the intuition and concept behind our method. Extensive experiments in the cross-sequence, cross-site, and cross-modality semi-supervised domain generalized medical image segmentation settings show the effectiveness of CausalAD and its superiority over the state-of-the-art. The code is available at https://github.com/Senyh/CausalAD.

Clustering spatio-temporal data in distributed systems is crucial for various applications such as traffic management, smart cities, telecommunications, and environmental monitoring. Despite the notable progress made in this field, several significant challenges persist: (a) in centralized systems, spatio-temporal data clustering necessitates that data be sent to the cloud for processing, which raises concerns about data transmission costs, latency, and privacy and security, (b) centralized systems incur high computational costs and require expensive hardware, resulting in prolonged runtime for algorithms, and (c) lack of well-defined space and time contiguous clusters adversely affects the overall usability of the clusters produced. These challenges are addressed by the proposed dnccVAT algorithm for assessing clustering tendency in spatio-temporal data within distributed systems, which is part of the visual assessment of clustering tendency family of algorithms. This algorithm effectively navigates the complexities associated with spatial-temporal relationships while minimizing communication overhead and ensuring scalability across distributed participant nodes. Extensive experiments were carried out on six real-world datasets, one of them being high-dimensional Big Data, comparing the proposed method with four state-of-the-art spatio-temporal data clustering algorithms and evaluating seven different performance measures to provide valuable insights into the effectiveness of the proposed approach.

Reinforcement learning (RL) has shown impressive results in sequential decision-making tasks. Large Language Models (LLMs) and Vision-Language Models (VLMs) have recently emerged, exhibiting impressive capabilities in multimodal understanding and reasoning. These advances have led to a surge of research integrating LLMs and VLMs into RL. This survey reviews representative works in which LLMs and VLMs are used to overcome key challenges in RL, such as lack of prior knowledge, long-horizon planning, and reward design. We present a taxonomy that categorizes these LLM/VLM-assisted RL approaches into three roles: agent, planner, and reward. We conclude by exploring open problems, including grounding, bias mitigation, improved representations, and action advice. By consolidating existing research and identifying future directions, this survey establishes a framework for integrating LLMs and VLMs into RL, advancing approaches that unify natural language and visual understanding with sequential decision-making.

Advancements in robotics have opened possibilities to automate tasks in various fields such as manufacturing, emergency response and healthcare. However, a significant challenge that prevents robots from operating in real-world environments effectively is out-of-distribution (OOD) situations, wherein robots encounter unforseen situations. One major OOD situations is when robots encounter faults, making fault adaptation essential for real-world operation for robots. Current state-of-the-art reinforcement learning algorithms show promising results but suffer from sample inefficiency, leading to low adaptation speed due to their limited ability to generalize to OOD situations. Our research is a step towards adding hardware fault tolerance and fast fault adaptability to machines. In this research, our primary focus is to investigate the efficacy of generative flow networks in robotic environments, particularly in the domain of machine fault adaptation. We simulated a robotic environment called Reacher in our experiments. We modify this environment to introduce four distinct fault environments that replicate real-world machines/robot malfunctions. The empirical evaluation of this research indicates that continuous generative flow networks (CFlowNets) indeed have the capability to add adaptive behaviors in machines under adversarial conditions. Furthermore, the comparative analysis of CFlowNets with reinforcement learning algorithms also provides some key insights into the performance in terms of adaptation speed and sample efficiency. Additionally, a separate study investigates the implications of transferring knowledge from pre-fault task to post-fault environments. Our experiments confirm that CFlowNets has the potential to be deployed in a real-world machine and it can demonstrate adaptability in case of malfunctions to maintain functionality.

The progression of medical image analysis methodologies has significantly assisted fundus clinical decision-making, such as disease diagnosis and lesion segmentation. However, low-quality fundus images bring a series of challenges to the automatic screening of diseases and the segmentation of lesions. Most existing methods primarily concentrate on enhancing image quality by utilizing the supervision of paired fundus images, which are difficult to collect in real medical applications. High-quality reference images are essential for guiding quality enhancement. To this end, we propose an enhancement method for low-quality fundus images, called RF-IQE, to alleviate the requirement for paired training images and only requires low-quality fundus images. Specifically, we first construct the patch-level high-/low-quality domains by employing a rule-based quality assessment scheme. Then, to achieve the fundus image quality enhancement and unified illumination styles simultaneously, we formulate them as a patch quality domain adaptation and a multi-style domain adaptation, respectively. We qualitatively and quantitatively demonstrate that our reference-free image quality enhancement network outperforms the conventional methods and exhibits comparable performance than the deep learning-based image enhancement methods with paired images on both the EyeQ and Messidor datasets. Furthermore, we also investigate the influence of the RF-IQE method on various fundus imaging analysis tasks, including vessel segmentation, optic disc segmentation, lesion segmentation, and disease classification.

Brain functional connectivity analysis is important for understanding brain development and brain disorders. Recent studies have suggested that the variations of functional connectivity among multiple subnetworks are closely related to the development of diseases. However, the existing works failed to sufficiently capture the complex correlation patterns among the subnetworks and ignored the learning of heterogeneous structural information across the subnetworks. To address these issues, we formulate a new paradigm for constructing and analyzing high-order heterogeneous functional brain networks via meta-paths and propose a Heterogeneous Graph representation Learning framework (BrainHGL). Our framework consists of three key aspects: 1) Meta-path encoding for capturing rich heterogeneous topological information, 2) Meta-path interaction for exploiting complex association patterns among subnetworks and 3) Meta-path aggregation for better meta-path fusion. To the best of our knowledge, we are the first to formulate the heterogeneous brain networks for better exploiting the relationship between the subnetwork interactions and the mental disease We evaluate BrainHGL on the private center Nanjing Medical University dataset (center NMU) and the public Autism Brain Imaging Data Exchange (ABIDE) dataset. We demonstrate the effectiveness of the proposed model across various disease classification tasks, including major depression disorder (MDD), bipolar disorder (BD) and autism spectrum disorder (ASD) diagnoses. In addition, our model provides deeper insights into disease interpretability, including the critical brain subnetwork connectivities, brain regions and functional pathways. We also identified disease subtypes consistent with previous neuroscientific studies by our model, which benefits the disease identification performance. The code is available at https://github.com/IntelliDAL/Graph/BrainHGL.

Medical images acquired under suboptimal conditions often suffer from quality degradation, such as low-light, blurring, and artifacts. Such degradations obscure the lesions and anatomical structures in medical images, making it difficult to distinguish key pathological regions. This significantly increases the risk of misdiagnosis by automated medical diagnostic systems or clinicians. To address this challenge, we propose a multi-Color space-based quality enhancement network (MSQNet) that effectively eliminates global low-quality factors while preserving pathology-related characteristics for improved clinical observation and analysis. We first revisit the properties of image quality enhancement in different color spaces, where the V-channel in the HSV space can better represent the contrast and brightness enhancement process, whereas the A/B-channel in the LAB space is more focused on the color change of low-quality images. The proposed framework harnesses the unique properties of different color spaces to optimize the image enhancement process. Specifically, we propose a pathology-preserving transformer, designed to selectively aggregate features across different color spaces and enable comprehensive multiscale feature fusion. Leveraging these capabilities, MSQNet effectively enhances low-quality RGB medical images while preserving key pathological features, thereby establishing a new paradigm in medical image enhancement. Extensive experiments on three public medical image datasets demonstrate that MSQNet outperforms traditional enhancement techniques and state-of-the-art methods, in terms of both quantitative metrics and qualitative visual assessment. MSQNet successfully improves image quality while preserving pathological features and anatomical structures, facilitating accurate diagnosis and analysis by medical professionals and automated systems.

Graph similarity estimation is a challenging task due to the complex graph structures. Though important and well-studied, three critical aspects are yet to be fully handled in a unified framework: 1) how to learn richer cross-graph interactions from a pairwise node perspective; 2) how to map the similarity matrix into a similarity score by exploiting the inherent structure in the similarity matrix; and 3) how to establish a self-supervised learning mechanism for graph similarity learning. To solve these issues, we explore multiple attention and self-supervised mechanisms for graph similarity learning in this work. More specifically, we propose a unified self-supervised nodewise attention-guided graph similarity learning framework (SNA-GSL) involving: 1) a correlation-guided contrastive learning for capturing valuable node embeddings and 2) a graph similarity learning for predicting similarity scores with multiple proposed attention mechanisms. Extensive experimental results on graph-graph regression task and graph classification task demonstrate that the proposed SNA-GSL performs favorably against state-of-the-art methods. Moreover, the remarkable achievement of our model in the graph classification task is a clear indication of its exceptional generalization capabilities. The code is available at https://github.com/IntelliDAL/Graph/SNA-GSL.

Text style transfer, an important research direction in natural language processing, aims to adapt the text to various preferences but often faces challenges with limited resources. In this work, we introduce a novel method termed Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning (SETTP) for effective style transfer in low-resource scenarios. First, SETTP learns source style-level prompts containing fundamental style characteristics from high-resource style transfer. During training, the source style-level prompts are transferred through an attention module to derive a target style-level prompt for beneficial knowledge provision in low-resource style transfer. Additionally, we propose instance-level prompts obtained by clustering the target resources based on the semantic content to reduce semantic bias. We also propose an automated evaluation approach of style similarity based on alignment with human evaluations using ChatGPT-4. Our experiments across three resourceful styles show that SETTP requires only 1/20th of the data volume to achieve performance comparable to state-of-the-art methods. In tasks involving scarce data like writing style and role style, SETTP outperforms previous methods by 16.24%.

Text style transfer (TST) is crucial in natural language processing, aiming to endow text with a new style without altering its meaning.In real-world scenarios, not all styles have abundant resources.This work introduces TWIST (reusing Transferable Weight Increments for Style Text generation), a novel framework to mitigate data scarcity by utilizing style features in weight increments to transfer low-resource styles effectively.During target style learning, we derive knowledge via a specially designed weight pool and initialize the parameters for the unseen style.To enhance the effectiveness of merging, the target style weight increments are often merged from multiple source style weight increments through singular vectors.Considering the diversity of styles, we also designed a multi-key memory network that simultaneously focuses on task-and instance-level information to derive the most relevant weight increments.Results from multiple style transfer datasets show that TWIST demonstrates remarkable performance across different backbones, achieving particularly effective results in low-resource scenarios.

The dynamic brain network learning methods ignored the separation of redundant disease-irrelevant information, resulting in the model only achieving suboptimal diagnosis results. Meanwhile, the supervised learning scheme inevitably suffers from poor generalization due to the limited data. To address these problems, we propose a Self-supervised Dynamic Brain network Disentangled representation learning framework named SDBD, which incorporates 1) a dynamic topology-aware encoder for capturing diverse topological information, 2) a cross decoder for reconstructing the graph structure and 3) a spatio-temporal learning model based on the multi-head self-attention mechanism for classification. To disentangle the disease-related information from the dynamic brain networks, we design a temporal contrastive loss and a structure reconstruction loss. We evaluate our model on three real-world mental diseases: Autism Spectrum Disorder (ASD), Major Depressive Disorder (MDD), and Bipolar Disorder (BD). The results indicate significant improvements in our SDBD over the state-of-the-art methods owing to the disentangled disease-related information. Moreover, our method can identify the biomarkers associated with the diseases, which is consistent with the previous studies. To the best of our knowledge, our work is the first attempt to disentangle the disease-related information for the dynamic brain network analysis. The code is available at https://github.com/IntelliDAL/Graph/tree/main/SDBD.

\beginabstract Dynamic graphs play an important role in many fields like social relationship analysis, recommender systems and medical science, as graphs evolve over time. It is fundamental to capture the evolution patterns for dynamic graphs. Existing works mostly focus on constraining the temporal smoothness between neighbor snapshots, however, fail to capture sharp shifts, which can be beneficial for graph dynamics embedding. To solve it, we assume the evolution of dynamic graph nodes can be split into temporal shift embedding and temporal consistency embedding. Thus, we propose the Self-supervised Temporal-aware Dynamic Graph representation Learning framework (STDGL) for disentangling the temporal shift embedding from temporal consistency embedding via a well-designed auxiliary task from the perspectives of both node local and global connectivity modeling in a self-supervised manner, further enhancing the learning of interpretable graph representations and improving the performance of various downstream tasks. Extensive experiments on link prediction, edge classification and node classification tasks demonstrate STDGL successfully learns the disentangled temporal shift and consistency representations. Furthermore, the results indicate significant improvements in our STDGL over the state-of-the-art methods, and appealing interpretability and transferability owing to the disentangled node representations. \endabstract

Topic modeling aims to discover latent themes in collections of text documents. It has various applications across fields such as sociology, opinion analysis, and media studies. In such areas, it is essential to have easily interpretable, diverse, and coherent topics. An efficient topic modeling technique should accurately identify flat and hierarchical topics, especially useful in disciplines where topics can be logically arranged into a tree format. In this paper, we propose Community Topic, a novel algorithm that exploits word co-occurrence networks to mine communities and produces topics. We also evaluate the proposed approach using several metrics and compare it with usual baselines, confirming its good performances. Community Topic enables quick identification of flat topics and topic hierarchy, facilitating the on-demand exploration of sub- and super-topics. It also obtains good results on datasets in different languages.

Medical image analysis techniques have been employed in diagnosing and screening clinical diseases. However, both poor medical image quality and illumination style inconsistency increase uncertainty in clinical decision-making, potentially resulting in clinician misdiagnosis. The majority of current image enhancement methods primarily concentrate on enhancing medical image quality by leveraging high-quality reference images, which are challenging to collect in clinical applications. In this study, we address image quality enhancement within a fully self-supervised learning setting, wherein neither high-quality images nor paired images are required. To achieve this goal, we investigate the potential of self-supervised learning combined with domain adaptation to enhance the quality of medical images without the guidance of high-quality medical images. We design a Domain Adaptation Self-supervised Quality Enhancement framework, called DASQE. More specifically, we establish multiple domains at the patch level through a designed rule-based quality assessment scheme and style clustering. To achieve image quality enhancement and maintain style consistency, we formulate the image quality enhancement as a collaborative self-supervised domain adaptation task for disentangling the low-quality factors, medical image content, and illumination style characteristics by exploring intrinsic supervision in the low-quality medical images. Finally, we perform extensive experiments on six benchmark datasets of medical images, and the experimental results demonstrate that DASQE attains state-of-the-art performance. Furthermore, we explore the impact of the proposed method on various clinical tasks, such as retinal fundus vessel/lesion segmentation, nerve fiber segmentation, polyp segmentation, skin lesion segmentation, and disease classification. The results demonstrate that DASQE is advantageous for diverse downstream image analysis tasks.

Over the last few years, associative classifiers have shown massive success in mining patterns using association rules. These rule-based classifiers offer a level of human interpretability, addressing a common concern stemming from several deep learning models. Various associative classifiers have been proposed over the past that have shown state-of-the-art performance. However, those classifiers suffer the limitation of requiring parametric values which vary across different datasets. Furthermore, those frameworks do not consider the statistical significance of the rules. Recently, some works have addressed this limitation by proposing an associative classifier that incorporates the idea of using statistical significance to mine association classification rules. Though the recent associative classifiers show good performance, their performance is greatly affected by the dimension of the data. In this study, we explore the weakness of the recent associative classification models and experiment with using ensemble models to overcome such limitations, particularly on aggregating the ensemble models in a concise but effective predictor. We use 10 UCI datasets for evaluation of our new approach. From our study, we find the results based on the ensemble model with a delayed pruning are very competitive and can better handle large dimensional data spaces.

Community detection and community search are both critical tasks in graph mining, each serving unique purposes and presenting distinct challenges. The former aims to partition the graph vertices into densely connected subsets, while the latter adopts a more ego-centric approach, focusing on a specific node or group of nodes to identify a densely-connected subgraph that contains these query nodes. However, many real-world networks are characterized by uncertainty, leading to the notion of uncertain or probabilistic graphs. The transition from deterministic graphs to uncertain graphs introduces new challenges. We present USIWO, an efficient and practical solution for community search in unweighted uncertain graphs with edge uncertainty. In addition to being accurate, the approach utilizes an efficient data structure for storing only the relevant parts of the network in main memory, eliminating the need to store the entire graph, making it a valuable tool in finding the core of a community on very large uncertain graphs, when there is limited time and memory available. The algorithm operates through a one-node-expansion approach, based on the concepts of strong and weak links within a graph. Experimental results on several datasets demonstrate the algorithm's efficiency and performance.

Consistency regularization and pseudo labeling-based semi-supervised methods perform co-training using the pseudo labels from multi-view inputs. However, such co-training models tend to converge early to a consensus, degenerating to the self-training ones, and produce low-confidence pseudo labels from the perturbed inputs during training. To address these issues, we propose an Uncertainty-guided Collaborative Mean-Teacher (UCMT) for semi-supervised semantic segmentation with the high-confidence pseudo labels. Concretely, UCMT consists of two main components: 1) collaborative mean-teacher (CMT) for encouraging model disagreement and performing co-training between the sub-networks, and 2) uncertainty-guided region mix (UMIX) for manipulating the input images according to the uncertainty maps of CMT and facilitating CMT to produce high-confidence pseudo labels. Combining the strengths of UMIX with CMT, UCMT can retain model disagreement and enhance the quality of pseudo labels for the co-training segmentation. Extensive experiments on four public medical image datasets including 2D and 3D modalities demonstrate the superiority of UCMT over the state-of-the-art. Code is available at: https://github.com/Senyh/UCMT.

We report on implementing MIRA, a mental health resource chatbot to support healthcare workers in finding timely and relevant mental health resources. To generate appropriate queries to our carefully curated resource database, the chatbot must correctly identify the intents of an interlocutor and extract relevant entities from the conversation. With insufficient labelled examples, we employ data augmentation to generate training data automatically. Moreover, instead of detecting intent and extracting entities independently with two different classifiers, we integrate the two tasks by taking advantage of their interdependencies obtaining 99% accuracy for intent detection and 95.4% accuracy in entity extraction.

Retinal fundus images have been applied for the diagnosis and screening of eye diseases, such as Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). However, both low-quality fundus images and style inconsistency potentially increase uncertainty in the diagnosis of fundus disease and even lead to misdiagnosis by ophthalmologists. Most of the existing fundus image enhancement methods mainly focus on improving the image quality by leveraging the guidance of high-quality images, which is difficult to be collected in medical applications. In this paper, we tackle image quality enhancement in a fully unsupervised setting, i.e., neither paired images nor high-quality images. To this end, we explore the potential of the self-supervised task for improving the quality of fundus images without the requirement of high-quality reference images, and proposed a Domain Adaptation Self-supervised Quality Enhancement framework, named DASQE. Specifically, we construct multiple patch-wise domains via a well-designed rule-based quality assessment scheme and style clustering. To achieve robust low-quality image enhancement and address style inconsistency, we formulate two self-supervised domain adaptation tasks to disentangle the features of image content, low-quality factors and style information by exploring intrinsic supervision signals within the low-quality images. Extensive experiments are conducted on four benchmark datasets, and results show that our DASQE method achieves new state-of-the-art performance when only low-quality images are available.

Diabetic retinopathy (DR) is one of the most serious complications of diabetes and is a prominent cause of permanent blindness. However, the low-quality fundus images increase the uncertainty of clinical diagnosis, resulting in a significant decrease on the grading performance of the fundus images. Therefore, enhancing the image quality is essential for predicting the grade level in DR diagnosis. In essence, we are faced with three challenges: (I) How to appropriately evaluate the quality of fundus images; (II) How to effectively enhance low-quality fundus images for providing reliable fundus images to ophthalmologists or automated analysis systems; (III) How to jointly train the quality assessment and enhancement for improving the DR grading performance. Considering the importance of image quality assessment and enhancement for DR grading, we propose a collaborative learning framework to jointly train the subnetworks of the image quality assessment as well as enhancement, and DR disease grading in a unified framework. The key contribution of the proposed framework lies in modelling the potential correlation of these tasks and the joint training of these subnetworks, which significantly improves the fundus image quality and DR grading performance. Our framework is a general learning model, which may be useful in other medical images with low-quality data. Extensive experimental results have shown that our method outperforms state-of-the-art DR grading methods by a considerable 73.6% ACC/71.2% Kappa and 88.5% ACC/86.3% Kappa on Messidor and EyeQ benchmark datasets, respectively. In addition, our method significantly enhances the low-quality fundus images while preserving fundus structure features and lesion information. To make the framework more general, we also evaluate the enhancement results in more downstream tasks, such as vessel segmentation.

The less training data and insufficient supervision limit the performance of the deep supervised models for brain disease diagnosis. It is significant to construct a learning framework that can capture more information in limited data and insufficient supervision. To address these issues, we focus on self-supervised learning and aim to generalize the self-supervised learning to the brain networks, which are non-Euclidean graph data. More specifically, we propose an ensemble masked graph self-supervised framework named BrainGSLs, which incorporates 1) a local topological-aware encoder that takes the partially visible nodes as input and learns these latent representations, 2) a node-edge bi-decoder that reconstructs the masked edges by the representations of both the masked and visible nodes, 3) a signal representation learning module for capturing temporal representations from BOLD signals and 4) a classifier used for the classification. We evaluate our model on three real medical clinical applications: diagnosis of Autism Spectrum Disorder (ASD), diagnosis of Bipolar Disorder (BD) and diagnosis of Major Depressive Disorder (MDD). The results suggest that the proposed self-supervised training has led to remarkable improvement and outperforms state-of-the-art methods. Moreover, our method is able to identify the biomarkers associated with the diseases, which is consistent with the previous studies. We also explore the correlation of these three diseases and find the strong association between ASD and BD. To the best of our knowledge, our work is the first attempt of applying the idea of self-supervised learning with masked autoencoder on the brain network analysis.

How do we perform efficient inference while retaining high translation quality? Existing neural machine translation models, such as Transformer, achieve high performance, but they decode words one by one, which is inefficient. Recent non-autoregressive translation models speed up the inference, but their quality is still inferior. In this work, we propose DSLP, a highly efficient and high-performance model for machine translation. The key insight is to train a non-autoregressive Transformer with Deep Supervision and feed additional Layer-wise Predictions. We conducted extensive experiments on four translation tasks (both directions of WMT'14 EN-DE and WMT'16 EN-RO). Results show that our approach consistently improves the BLEU scores compared with respective base models. Specifically, our best variant outperforms the autoregressive model on three translation tasks, while being 14.8 times more efficient in inference.

Most recent semantic segmentation methods adopt a U-Net framework with an encoder-decoder architecture. It is still challenging for U-Net with a simple skip connection scheme to model the global multi-scale context: 1) Not each skip connection setting is effective due to the issue of incompatible feature sets of encoder and decoder stage, even some skip connection negatively influence the segmentation performance; 2) The original U-Net is worse than the one without any skip connection on some datasets. Based on our findings, we propose a new segmentation framework, named UCTransNet (with a proposed CTrans module in U-Net), from the channel perspective with attention mechanism. Specifically, the CTrans (Channel Transformer) module is an alternate of the U-Net skip connections, which consists of a sub-module to conduct the multi-scale Channel Cross fusion with Transformer (named CCT) and a sub-module Channel-wise Cross-Attention (named CCA) to guide the fused multi-scale channel-wise information to effectively connect to the decoder features for eliminating the ambiguity. Hence, the proposed connection consisting of the CCT and CCA is able to replace the original skip connection to solve the semantic gaps for an accurate automatic medical image segmentation. The experimental results suggest that our UCTransNet produces more precise segmentation performance and achieves consistent improvements over the state-of-the-art for semantic segmentation across different datasets and conventional architectures involving transformer or U-shaped framework. Code: https://github.com/McGregorWwww/UCTransNet.

Associative classifiers have shown competitive performance with state-of-the-art methods for predicting class labels. In addition to accuracy performance, associative classifiers produce human readable rules for classification which provides an easier way to understand the decision process of the model. Early models of associative classifiers suffered from the limitation of selecting proper threshold values which are dataset specific. Recent work on associative classifiers eliminates that restriction by searching for statistically significant rules. However, a high dimensional feature vector in the training data impacts the performance of the model. Ensemble models like Random Forest are also very powerful tools for classification but the decision process of Random Forest is not easily understandable like the associative classifiers. In this study we propose Dynamic Ensemble Associative Learning (DEAL) where we use associative classifiers as base learners on feature sub-spaces. In our approach we select a subset of the feature vector to train each of the base learners. Instead of a random selection, we propose a dynamic feature sampling procedure which automatically defines the number of base learners and ensures diversity and completeness among the subset of feature vectors. We use 10 datasets from the UCI repository and evaluate the performance of the model in terms of accuracy and memory requirement. Our ensemble approach using the proposed sampling method largely decreases the memory requirement in the case of datasets having a large number of features and this without jeopardising accuracy. In fact, accuracy is also improved in most cases. Moreover, the decision process of our DEAL approach remains human interpretable by collecting and ranking the rules generated by the base learners predicting the final class label.

We present our novel, hyperparameter-free topic modelling algorithm, Community Topic. Our algorithm is based on mining communities from term co-occurrence networks. We empirically evaluate and compare Community Topic with Latent Dirichlet Allocation and the recently developed top2vec algorithm. We find that Community Topic runs faster than the competitors and produces topics that achieve higher coherence scores. Community Topic can discover coherent topics at various scales. The network representation used by Community Topic results in a natural relationship between topics and a topic hierarchy. This allows sub- and super-topics to be found on demand. These features make Community Topic the ideal tool for downstream applications such as applied research and conversational agents.

Modern approaches in Natural Language Processing (NLP) require, ideally, large amounts of labelled data for model training. However, new language resources, for example, for Named Entity Recognition (NER), Co-reference Resolution (CR), Entity Linking (EL) and Relation Extraction (RE), naming a few of the most popular tasks in NLP, have always been challenging to create since manual text annotations can be very time-consuming to acquire. While there may be an acceptable amount of labelled data available for some of these tasks in one language, there may be a lack of datasets in another. WEXEA is a tool to exhaustively annotate entities in the English Wikipedia. Guidelines for editors of Wikipedia articles result, on the one hand, in only a few annotations through hyperlinks, but on the other hand, make it easier to exhaustively annotate the rest of these articles with entities than starting from scratch. We propose the following main improvements to WEXEA: Creating multi-lingual corpora, improved entity annotations using a proven NER system, annotating dates and times. A brief evaluation of the annotation quality of WEXEA is added.

Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, Siva Reddy. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Abstract The goal of information-seeking dialogue is to respond to seeker queries with natural language utterances that are grounded on knowledge sources. However, dialogue systems often produce unsupported utterances, a phenomenon known as hallucination. To mitigate this behavior, we adopt a data-centric solution and create FaithDial, a new benchmark for hallucination-free dialogues, by editing hallucinated responses in the Wizard of Wikipedia (WoW) benchmark. We observe that FaithDial is more faithful than WoW while also maintaining engaging conversations. We show that FaithDial can serve as training signal for: i) a hallucination critic, which discriminates whether an utterance is faithful or not, and boosts the performance by 12.8 F1 score on the BEGIN benchmark compared to existing datasets for dialogue coherence; ii) high-quality dialogue generation. We benchmark a series of state-of-the-art models and propose an auxiliary contrastive objective that achieves the highest level of faithfulness and abstractiveness based on several automated metrics. Further, we find that the benefits of FaithDial generalize to zero-shot transfer on other datasets, such as CMU-Dog and TopicalChat. Finally, human evaluation reveals that responses generated by models trained on FaithDial are perceived as more interpretable, cooperative, and engaging.

Modeling spatio-temporal dynamics in functional brain networks is critical for underlying the functional mechanism of autism spectrum disorder (ASD). In our study, we propose an end-to-end framework called temporal graph representation learning for brain networks, which thoroughly captures spatio-temporal features in resting-state functional magnetic resonance imaging (rs-fMRI) data. Specifically, we first transform rs-fMRI time-series into temporal multi-graph using a sliding window technique. A temporal multi-graph clustering is then designed to eliminate the inconsistency of the temporal multi-graph series. Then, a graph structure aware LSTM (GSA-LSTM) is proposed to capture the spatio-temporal embedding for temporal graphs. The proposed GSA-LSTM can not only capture discriminative features for prediction but also impute the incomplete graphs for the temporal multi-graph series. Extensive experiments on autism brain imaging data exchange (ABIDE) dataset shows the effectiveness of our proposed framework. The results demonstrate that the proposed dynamic brain network embedding learning outperforms the state of-the-art brain network classification models. Furthermore, the obtained clustering results are consistent with the previous neuroimaging-derived evidence of biomarkers for autism spectrum disorder (ASD).

Explicitly modeling emotions in dialogue generation has important applications, such as building empathetic personal companions. In this study, we consider the task of expressing a specific emotion for dialogue generation. Previous approaches take the emotion as a training signal, which may be ignored during inference. Here, we propose a search-based emotional dialogue system by simulated annealing (SA). Specifically, we first define a scoring function that combines contextual coherence and emotional correctness. Then, SA iteratively edits a general response, and search for a generation with a high score. In this way, we enforce the presence of the desired emotion. We evaluate our system on the NLPCC2017 dataset. The proposed method shows about 12% improvements in emotion accuracy compared with the previous state-of-the-art method, without hurting the generation quality (measured by BLEU).

Dialogue systems powered by large pretrained language models exhibit an innate ability to deliver fluent and natural-sounding responses.Despite their impressive performance, these models are fitful and can often generate factually incorrect statements impeding their widespread adoption.In this paper, we focus on the task of improving faithfulness and reducing hallucination of neural dialogue systems to known facts supplied by a Knowledge Graph (KG).We propose NEU-RAL PATH HUNTER which follows a generatethen-refine strategy whereby a generated response is amended using the KG.NEURAL PATH HUNTER leverages a separate tokenlevel fact critic to identify plausible sources of hallucination followed by a refinement stage that retrieves correct entities by crafting a query signal that is propagated over a k-hop subgraph.We empirically validate our proposed approach on the OpenDialKG dataset (Moon et al., 2019) against a suite of metrics and report a relative improvement of faithfulness over dialogue responses by 20.35% based on FeQA (Durmus et al., 2020).The code is available at https://github.com/ nouhadziri/Neural-Path-Hunter.

We focus on exploring various approaches of Zero-Shot Learning (ZSL) and their explainability for a challenging yet important supervised learning task, notorious for training data scarcity, i.e. Depression Symptoms Detection (DSD) from text. We start with a comprehensive synthesis of different components of our ZSL modelling and analysis of our ground truth samples and Depression symptom clues curation process with the help of a practicing Clinician. We next analyze the accuracy of various state-of-the-art ZSL models and their potential enhancements for our task. Further, we sketch a framework for the use of ZSL for hierarchical text-based explanation mechanism, which we call, Syntax Tree-Guided Semantic Explanation (STEP). Finally, we summarize experiments from which we conclude that we can use ZSL models and achieve reasonable accuracy and explainability, measured by a proposed Explainability Index (EI). This work is, to our knowledge, the first work to exhaustively explore the efficacy of ZSL models for DSD task, both in terms of accuracy and explainability.

Most major events are often accompanied by misinformation on online Social Networking platforms. Due to its nature, the COVID-19 pandemic was bound to lead to an explosion of information online, much of it false or misleading. This information explosion, termed "infodemic" by the World Health Organization (WHO), has revealed the need for automatic fake news detection to help with the exponentially growing flow of unverified information. The objective of this study is to explore combinations of different supervised classification models trained on different general and domain-specific embeddings, and compare the effects of the iterations on the results. We also analyze the results to determine whether the differences in weighted F1-score performance metrics are statistically significant. Ultimately, we demonstrate that concatenation of general and context-specific embeddings improves performance. Our research shows promise for health misinformation detection and formulation of effective public health responses.

Chenyang Huang, Amine Trabelsi, Xuebin Qin, Nawshad Farruque, Lili Mou, Osmar Zaïane. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

MRI brain segmentation plays an important part in computer-aided diagnosis, which visually reveals the changes in brain structure for doctors to quickly and accurately discover and treat diseases related to brain tissue morphology. The fuzzy C-means (FCM) algorithm performs well when the segmenting images with no noise and with intensity uniformity. However, the MRI brain images are always defective in noise and intensity nonuniformity and thus we propose a novel FCM algorithm named adaptive FCM with neighborhood membership (FCM_anm). We design a filtering process with neighborhood membership to reduce the negative influence of noise and a novel objective function which further considers the spatial membership information adaptively. Finally, to verify the performance of our method, several experiments comparing among the Experimental results demonstrate the proposed method consistently outperforms the state-of-the-art FCM-based algorithms in synthetic images, simulated and real brain MR images with effects of the noise and intensity non-uniformity.

Diabetic retinopathy (DR) is one of the most concerning, common and serious diseases in the ophthalmology community. Early detection and treatment of DR can significantly reduce the risk of vision loss in patients. Traditional DR automatic classification algorithms rely on the precise detection of microaneurysms (MA) and hemorrhage (H) lesions. Such lesion annotation is an expensive and time-consuming process, hence it is expected to develop automatic grading methods with only image-level annotations. The lack of the position of MA and H hinders the traditional supervised algorithms for the accurate identification. In our work, we formulate the weakly supervised DR grading as a multi-instance learning problem, and propose a domain adaptation multi-instance learning with attention mechanism for DR grading. Specifically, labeled instances are generated by cross-domain to filter irrelevant instances in the target domain. To model the relationship between the suspicious instances and bag label, a multi-instance learning with attention mechanism is developed to acquire the location information of highly suspected lesions and predict the grade of DR. We evaluate our proposed algorithm on the Messidor dataset, and the experimental results demonstrate that it achieves an average accuracy of 0.764 and an AUC value of 0.749 respectively, outperforming state-of-the-art approaches.

We introduce a novel efficient approach for community detection based on a formal definition of the notion of community. We name the links that run between communities weak links and links being inside communities strong links. We put forward a new objective function, called SIWO (Strong Inside, Weak Outside) which encourages adding strong links to the communities while avoiding weak links. This process allows us to effectively discover communities in social networks without the resolution and field of view limit problems some popular approaches suffer from. The time complexity of this new method is linear in the number of edges. We demonstrate the effectiveness of our approach on various real and artificial datasets with large and small communities.

The majority of research on community detection in attributed networks follows an “early fusion” approach, in which the structural and attribute information about the network are integrated together as the guide to community detection. In this paper, we propose an approach called late-fusion , which looks at this problem from a different perspective. We first exploit the network structure and node attributes separately to produce two different partitionings. Later on, we combine these two sets of communities via a fusion algorithm, where we introduce a parameter for weighting the importance given to each type of information: node connections and attribute values. Extensive experiments on various real and synthetic networks show that our late-fusion approach can improve detection accuracy from using only network structure. Moreover, our approach runs significantly faster than other attributed community detection algorithms including early fusion ones.

Building predictive models for information extraction from text, such as named entity recognition or the extraction of semantic relationships between named entities in text, requires a large corpus of annotated text. Wikipedia is often used as a corpus for these tasks where the annotation is a named entity linked by a hyperlink to its article. However, editors on Wikipedia are only expected to link these mentions in order to help the reader to understand the content, but are discouraged from adding links that do not add any benefit for understanding an article. Therefore, many mentions of popular entities (such as countries or popular events in history), or previously linked articles, as well as the article’s entity itself, are not linked. In this paper, we discuss WEXEA, a Wikipedia EXhaustive Entity Annotation system, to create a text corpus based on Wikipedia with exhaustive annotations of entity mentions, i.e. linking all mentions of entities to their corresponding articles. This results in a huge potential for additional annotations that can be used for downstream NLP tasks, such as Relation Extraction. We show that our annotations are useful for creating distantly supervised datasets for this task. Furthermore, we publish all code necessary to derive a corpus from a raw Wikipedia dump, so that it can be reproduced by everyone.

Previous work has shown that machine learning algorithms lend themselves to clinical decision-making and are a valuable tool for physicians. For clinical data, it is often necessary to assign multiple labels to a patient record by choosing from a large number of potential labels. A key problem in learning from multi-labelled data is how to exploit the information contained in the correlations between labels. The hypergraph-based multi-label learning method learns from data by exploiting the spectral property of the hypergraph that encodes the correlation structure of labels. However, the problem with this method is the difficulty with which interpretations can be made. This is mainly due to its inability to recognize the importance of key features in the original feature space. Moreover, it is hard to comprehensively capture the complex structure of the correlations between labels. To overcome these difficulties and improve interpretability, we propose an l21-norm regularized Graph Laplacian multi-label learning to perform feature selection and label embedding simultaneously. In-depth experimental studies, using the publicly available Medical Information Mart for Intensive Care (MIMIC-III) database, validate the effectiveness of our approach.

Machine learning algorithms and multivariate data analysis methods have been widely utilized in the field of Alzheimer's disease (AD) research in recent years. Predicting cognitive performance of subjects from neuroimage measures and identifying relevant imaging biomarkers are important research topics in the study of Alzheimer's disease. Multi-task based feature learning (MTFL) have been widely studied to select a discriminative feature subset from MRI features, and improve the performance by incorporating inherent correlations among multiple clinical cognitive measures. It is known that the brain imaging measures are often correlated with each other, and AD is closely related to the inter-correlation among different brain regions. However, the multi-task based feature learning (MTFL) method neglects the inherent correlation among brain imaging measures. We present a novel regularized multi-task learning approach via a joint sparsity-inducing regularization to effectively incorporate both a relatedness among multiple cognitive score prediction tasks and a useful inherent correlation between brain imaging measures by exploiting correlations among features. It allows the simultaneous selection of a common set of biomarkers for all tasks and the preservation of the inherent structure of imaging measures. The reported experiments on the ADNI dataset show that the proposed method is effective and promising.

Online advertising is one of the most lucrative forms of advertising, making it an important channel of advertising media. Contextual Advertising is a type of online display advertising that takes cues from the content of the triggering page and displays advertisements that are relevant to the current context. However, on several occasions, the context may have a negative connotation, and displaying advertisements that are relevant to it might prove to be detrimental to the advertiser. We refer to such a scenario as an unfortunate placement. In this work, we propose APNEA (Ad Positive NEgative Analysis), a light-weight system that uses a sentiment-oriented approach to rank the advertisers such that positively correlated brands are ranked higher than brands that are neutral or negatively correlated. Experiments show that APNEA helps avoid unfortunate placements while maintaining ad-relevance. It outperforms several baselines in terms of accuracy on human-annotated test data while having a lower run-time, which is crucial for real-time bidding systems.

In this work, we focus on the problem of local community detection with edge uncertainty. We use an estimator to cope with the intrinsic uncertainty of the problem. Then we illustrate with an example that periphery nodes tend to be grouped into their neighbor communities in uncertain networks, and we propose a new measure K to address this problem. Due to the very limited publicly available uncertain network datasets, we also put forward a way to generate uncertain networks. Finally, we evaluate our algorithm using existing ground truth as well as based on common metrics to show the effectiveness of our proposed approach.

The class imbalance problem is a pervasive issue in many real-world domains. Oversampling methods that inflate the rare class by generating synthetic data are amongst the most popular techniques for resolving class imbalance. However, they concentrate on the characteristics of the minority class and use them to guide the oversampling process. By completely overlooking the majority class, they lose a global view on the classification problem and, while alleviating the class imbalance, may negatively impact learnability by generating borderline or overlapping instances. This becomes even more critical when facing extreme class imbalance, where the minority class is strongly underrepresented and on its own does not contain enough information to conduct the oversampling process. We propose a novel method for synthetic oversampling that uses the rich information inherent in the majority class to synthesize minority class data. This is done by generating synthetic data that is at the same Mahalanbois distance from the majority class as the known minority instances. We evaluate over 26 benchmark datasets, and show that our method offers a distinct performance improvement over the existing state-of-the-art in oversampling techniques.

In this ongoing work, we present the Grebe social data aggregation framework for extracting geo-fenced Twitter data for analysis of user engagement in health and wellness topics. Grebe also provides various visualization tools for analyzing temporal and geographical health trends. Grebe currently has over 18 million indexed public tweets, and is the first of its kind for Canadian researchers. The large dataset is used for analyzing three types of contexts: geographical context via prediction of user location using supervised learning, topical context via determining health-related tweets using various learning approaches, and affective context via sentiment analysis of tweets using rule-based methods. For the first, we define user location as the position from which users are posting a tweet and use standard precision metrics for evaluation with promising results for predicting provinces and cities from tweet text. For the second, we use a broader definition of health using the six dimensions of wellness model and evaluate using manually annotated documents with good results using supervised and semi-supervised machine learning. For the third, we use the indexed tweets to show current trends in emotions and opinions and demonstrate trends in polarity and emotions across various Canadian provinces. The combination of these contexts provides useful insights for digital epidemiology. Ultimately, the vision of Grebe is to provide researchers with Canada-specific social web datasets through an open source platform with an accessible RESTful API, and this paper showcases Grebe's potential and presents our progress towards achieving these goals.

A clustering agreement index quantifies the similarity between two given clusterings. It is most commonly used to compare the results obtained from different clustering algorithms against the ground-truth clustering in the benchmark datasets. In this paper, we present a general Clustering Agreement Index (CAI) for comparing disjoint and overlapping clusterings. CAI is generic and introduces a family of clustering agreement indexes. In particular, the two widely used indexes of Adjusted Rand Index (ARI), and Normalized Mutual Information (NMI), are special cases of the CAI. Our index, therefore, provides overlapping extensions for both these commonly used indexes, whereas their original formulations are only defined for disjoint cases. Lastly, unlike previous indexes, CAI is flexible and can be adapted to incorporate the structure of the data, which is important when comparing clusters in networks, a.k.a communities.

Assessing learning outcomes for students in higher education institutes is an interesting task with many potential applications for all involved stakeholders (students, administrators, potential employers, etc.). In this paper, we propose a course recommendation system for students based on the assessment of their "graduate attributes" (i.e. attributes that describe the developing values of students). Students rate the improvement in their graduating attributes after a course is finished and a collaborative filtering algorithm is utilized in order to suggest courses that were taken by fellow students and rated in a similar way. An extension to weigh the most recent ratings as more important is included in the algorithm which is shown to have better accuracy than the baseline approach. Experimental results using correlation thresholding and the nearest neighbors approach show that such a recommendation system can be effective when an active neighborhood of 10-15 students is used and show that the numbers of users used can be decreased effectively to one fourth of the whole population for improving the performance of the algorithm.

Relational learning exploits relationships among instances manifested in a network to improve the predictive performance of many network mining tasks. Due to its empirical success, it has been widely applied in myriad domains. In many cases, individuals in a network are highly idiosyncratic. They not only connect to each other with a composite of factors but also are often described by some content information of high dimensionality specific to each individual. For example in social media, as user interests are quite diverse and personal; posts by different users could differ significantly. Moreover, social content of users is often of high dimensionality which may negatively degrade the learning performance. Therefore, it would be more appealing to tailor the prediction for each individual while alleviating the issue related to the curse of dimensionality. In this paper, we study a novel problem of Personalized Relational Learning and propose a principled framework PRL to personalize the prediction for each individual in a network. Specifically, we perform personalized feature selection and employ a small subset of discriminative features customized for each individual and some common features shared by all to build a predictive model. On this account, the proposed personalized model is more human interpretable. Experiments on real-world datasets show the superiority of the proposed PRL framework over traditional relational learning methods.

Established associative classification algorithms have shown to be very effective in handling categorical data such as text data. The learned model is a set of rules that are easy to understand and can be edited. However, they still suffer from the following limitations: first, they mostly use the support-confidence framework to mine classification association rules which require the setting of some confounding parameters; second, the lack of statistical dependency in the used framework may lead to the omission of many interesting rules and the detection of meaningless rules; third, the rule generation process usually generates a sheer number of rules which puts in question the interpretability and readability of the learned associative classification model. In this paper, we propose a novel associative classifier, SigDirect, to address the above problems. In particular, we use Fisher’s exact test as a significance measure to directly mine classification association rules by some effective pruning strategies. Without any threshold settings like minimum support and minimum confidence, SigDirect is able to find non-redundant classification association rules which express a statistically significant dependency between a set of antecedent items and a consequent class label. To further reduce the number of noisy rules, we present an instance-centric rule pruning strategy to find a subset of rules of high quality. At last, we propose and investigate various rule classification strategies to achieve a more accurate classification model. Experimental results on real-world datasets show that SigDirect achieves better performance in terms of classification accuracy when measured with state-of-the-art rule based and associative classifiers. Furthermore, the number of rules generated by SigDirect is orders of magnitude smaller than the number of rules found by other associative classifiers, which is very appealing in practice.

Most real world applications comprise databases having multiple tables. It becomes further complicated in the realm of Big Data where related information is spread over different data repositories. However, data mining techniques are usually applied on a single flat table. This work focuses on generating a mining table by aggregating information from multiple local tables and external data sources and automatically generating potentially discriminant features. It extends data aggregation techniques by navigating paths where a single table is traversed multiple times. Such paths are not considered by existing techniques, which results in the loss of several attributes. Our framework also prevents leakage of the class information by avoiding features built after the knowledge of the class label. Experiments are performed on transactional data of a U.S. consumer electronics retailer to predict causes of product returns. In addition, we augmented the dataset with Suppliers information and Reviews to show the value of data integration. The results show that our technique improves classification accuracy and generates discriminant features that mitigate the impact of class imbalance.

Although statistical and machine learning methods require the input data to be in a tabular format, in real-world applications data are often stored across several tables in a relational database. How to build a single mining table from a relational database is a critical pre-processing step of any classification method, because including the right attributes may dramatically boost the accuracy of the classifier. We propose a methodology and implement a software program, Dataconda, to automatically mine a relational database. The user selects a class attribute contained in a table of the database and the procedure builds and selects predictors by exploring the whole database and aggregating information, without any user intervention. For example, our procedure may find that the best predictor for “product return” is the proportion of products returned by the same customer in the past, even if the user has not built any such attribute. Our procedure produces more expressive attributes than existing methods. Our experiments on the ISMS Durable Goods Datasets, a publicly available data set of product returns in retailing, suggest that our method allows new knowledge to emerge.

Rule-based classifier has shown its popularity in building many decision support systems such as medical diagnosis and financial fraud detection. One major advantage is that the models are human understandable and can be edited. Associative classifiers, as an extension of rule-based classifiers, use association rules to associate attributes with class labels. A delicate issue of associative classifiers is the need for subtle thresholds: minimum support and minimum confidence. Without prior knowledge, it could be difficult to choose the proper thresholds, and the discovered rules within the support-confidence framework are not statistically significant, i.e., inclusion of noisy rules and exclusion of valuable rules. Besides, most associative classifiers proposed so far, are built with only positive association rules. Negative rules, however, are also able to provide valuable information to discriminate between classes. To solve the above mentioned problems, we propose a novel associative classifier which is built upon both positive and negative classification association rules that show statistically significant dependencies. Experimental results on real-world datasets show that our method achieves competitive or even better performance than well-known rule-based and associative classifiers in terms of both classification accuracy and computational efficiency.

Anomaly detection in time series is one of the fundamental issues in data mining that addresses various problems in different domains such as intrusion detection in computer networks, irregularity detection in healthcare sensory data and fraud detection in insurance or securities. Although, there has been extensive work on anomaly detection, majority of the techniques look for individual objects that are different from normal objects but do not take the temporal aspect of data into consideration. We are particularly interested in contextual outlier detection methods for time series that are applicable to fraud detection in securities. This has significant impacts on national and international securities markets. In this paper, we propose a prediction-based Contextual Anomaly Detection (CAD) method for complex time series that are not described through deterministic models. The proposed method improves the recall from 7% to 33% compared to kNN and Random Walk without compromising the precision.

We explore methods for effectively extracting information from clinical narratives that are captured in a public health consulting phone service called HealthLink. Our research investigates the application of state-of-the-art natural language processing and machine learning to clinical narratives to extract information of interest. The currently available data consist of dialogues constructed by nurses while consulting patients by phone. Since the data are interviews transcribed by nurses during phone conversations, they include a significant volume and variety of noise. When we extract the patient-related information from the noisy data, we have to remove or correct at least two kinds of noise: explicit noise , which includes spelling errors, unfinished sentences, omission of sentence delimiters, and variants of terms, and implicit noise , which includes non-patient information and patient's untrustworthy information. To filter explicit noise, we propose our own biomedical term detection/normalization method: it resolves misspelling, term variations, and arbitrary abbreviation of terms by nurses. In detecting temporal terms, temperature, and other types of named entities (which show patients’ personal information such as age and sex), we propose a bootstrapping-based pattern learning process to detect a variety of arbitrary variations of named entities. To address implicit noise, we propose a dependency path-based filtering method. The result of our denoising is the extraction of normalized patient information, and we visualize the named entities by constructing a graph that shows the relations between named entities. The objective of this knowledge discovery task is to identify associations between biomedical terms and to clearly expose the trends of patients’ symptoms and concern; the experimental results show that we achieve reasonable performance with our noise reduction methods.

A social role is a special position an individual possesses within a network, which indicates his or her behaviours, expectations, and responsibilities. Identifying the roles that individuals play in a social network has various direct applications, such as detecting influential members, trustworthy people, idea innovators, etc. Roles can also be used for further analyses of the network, e.g. community detection, temporal event prediction, and summarization. In this paper, we propose a structural social role mining framework (SSRM), which is built to identify roles, study their changes, and analyze their impacts on the underlying social network. We define fundamental roles in a social network (namely leader, outermost, mediator, and outsider), and then propose methodologies to identify them, and track their changes. To identify these roles, we leverage the traditional social network analyses and metrics, as well as proposing new measures, including community-based variants for the Betweenness centrality. Our results indicate how the changes in the structural roles, in combination with the changes in the community structure of a network, can provide additional clues into the dynamics of networks.

We present our novel community mining algorithm that uses only local information to accurately identify communities, outliers, and hubs in social networks. The main component of our algorithm is the T metric, which evaluates the relative quality of a community by considering the number of internal and external triads (3-node cliques) it contains. Furthermore we propose an intuitive statistical method based on our T metric, which correctly identifies outlier and hub nodes within each discovered community. Finally, we evaluate our approach on a series of ground-truth networks and show that our method outperforms the state-of-the-art in community mining algorithms.

Finding patterns of interaction and predicting the future structure of networks has many important applications, such as recommendation systems and customer targeting. Community structure of social networks may undergo different temporal events and transitions. In this paper, we propose a framework to predict the occurrence of different events and transition for communities in dynamic social networks. Our framework incorporates key features related to a community - its structure, history, and influential members, and automatically detects the most predictive features for each event and transition. Our experiments on real world datasets confirms that the evolution of communities can be predicted with a very high accuracy, while we further observe that the most significant features vary for the predictability of each event and transition.

Market manipulation remains the biggest concern of investors in today's securities market, despite fast and strict responses from regulators and exchanges to market participants that pursue such practices. The existing methods in the industry for detecting fraudulent activities in securities market rely heavily on a set of rules based on expert knowledge. The securities market has deviated from its traditional form due to new technologies and changing investment strategies in the past few years. The current securities market demands scalable machine learning algorithms supporting identification of market manipulation activities. In this paper we use supervised learning algorithms to identify suspicious transactions in relation to market manipulation in stock market. We use a case study of manipulated stocks during 2003. We adopt CART, conditional inference trees, C5.0, Random Forest, Naïve Bayes, Neural Networks, SVM and kNN for classification of manipulated samples. Empirical results show that Naïve Bayes outperform other learning methods achieving F2measure of 53% (sensitivity and specificity are 89% and 83% respectively).

This work proposes an unsupervised method intended to enhance the quality of opinion mining in contentious text. It presents a Joint Topic Viewpoint (JTV) probabilistic model to analyse the underlying divergent arguing expressions that may be present in a collection of contentious documents. It extends the original Latent Dirichlet Allocation (LDA), which makes it domain and thesaurus-independent, e.g., does not rely on Word Net coverage. The conceived JTV has the potential of automatically carrying the tasks of extracting associated terms denoting an arguing expression, according to the hidden topics it discusses and the embedded viewpoint it voices. Furthermore, JTV's structure enables the unsupervised grouping of obtained arguing expressions according to their viewpoints, using a constrained clustering approach. Experiments are conducted on three types of contentious documents: polls, online debates and editorials. The qualitative and quantitative analysis of the experimental results show the effectiveness of our model to handle six different contentious issues when compared to a state-of-the-art method. Moreover, the ability to automatically generate distinctive and informative patterns of arguing expressions is demonstrated.

Class imbalance is one of the challenging problems for machine learning in many real-world applications. Other issues, such as within-class imbalance and high dimensionality, can exacerbate the problem. We propose a method HPS-DRS that combines two i

Interest in extracting information from biomedical documents has increased significantly in recent years but has always been challenged by the ambiguity of natural language. An important source of ambiguity is the usage of polysemous words: words with multiple meanings. Word sense disambiguation algorithms attempt to solve this problem by finding the correct meaning of a polysemous word in a given context, but very few algorithms were designed to disambiguate biomedical text. In this study we propose a word sense disambiguation algorithm focused on biomedical text. The proposed algorithm does not need to be trained and uses a relatively small knowledge base.

Social networks are usually drawn from the interactions between individuals, and therefore are temporal and dynamic in essence. Examining how the structure of these networks changes over time provides insights into their evolution patterns, factors that trigger the changes, and ultimately predict the future structure of these networks. One of the key structural characteristics of networks is their community structure --groups of densely interconnected nodes. Communities in a dynamic social network span over periods of time and are affected by changes in the underlying population, i.e. they have fluctuating members and can grow and shrink over time. In this paper, we introduce a new incremental community mining approach, in which communities in the current time are obtained based on the communities from the past time frame. Compared to previous independent approaches, this incremental approach is more effective at detecting stable communities over time. Extensive experimental studies on real datasets, demonstrate the applicability, effectiveness, and soundness of our proposed framework.

We explore methods for effectively extracting information from clinical narratives, which are captured in a public health consulting phone service called HealthLink. The currently available data consists of dialogues constructed by nurses while consulting patients on the phone. Since the data are interviews transcribed by nurses during phone conversations, they include a significant volume and variety of noise: First is explicit noise, which includes spelling errors, unfinished sentences, omission of sentence delimiters, variants of terms, etc. Second is implicit noise, which includes non-patient's information and negation of patient's information. To filter explicit noise, we propose our biomedical term detection/normalization method: it resolves misspelling, term variations, and arbitrary abbreviation of terms by nurses. In detecting temporal terms and other types of named entities (which show patients' personal information such as age, and sex), we propose a bootstrapping-based pattern learning to detect all kinds of arbitrary variations of the named entities. To address implicit noise, we propose a dependency path-based filtering method. The result of our denoising is the extraction of normalized patient information. The experimental results show that we achieve reasonable performance with our noise reduction methods.

Health discussion forums are popular, contain useful information, but are difficult to navigate given the high volume of discussions and the non-standard language used. To simplify exploring the medical discussion forums, we extract from natural language text medical entities using MeSH and use three methods to find relationships between the entities, then form an ontology of those entities as an effective interface for navigating through discussions.

Many lung nodule computer-aided detection methods have been proposed to help radiologists in their decision making. Because high sensitivity is essential in the candidate identification stage, there are countless false positives produced by the initial suspect nodule generation process, giving more work to radiologists. The difficulty of false positive reduction lies in the variation of the appearances of the potential nodules, and the imbalance distribution between the amount of nodule and non-nodule candidates in the dataset. To solve these challenges, we extend the random subspace method to a novel Cost Sensitive Adaptive Random Subspace ensemble (CSARS), so as to increase the diversity among the components and overcome imbalanced data classification. Experimental results show the effectiveness of the proposed method in terms of G-mean and AUC in comparison with commonly used methods.

In the medical field, we are amassing phenomenal amounts of data. Because of understandable ethical and legal responsibility to maintain the privacy, many techniques of anonymization have been proposed to provide means of data publishing without jeopardizing privacy. The strictness of the techniques is putting in question the utility of the health data after severe anonymization.

Word Sense Disambiguation is the task of automatically identifying the correct sense of an ambiguous word. Biomedical documents, similar to other narrative documents, suffer from ambiguity, which impacts the ability to automatically extract knowledge contained in the document text. In this study, we propose a graph-based word sense disambiguation algorithm focused on biomedical text. The proposed algorithm uses the UMLS Metathesaurus as its source of knowledge. Evaluation is carried out using the MSH-WSD data set.

The performance of traditional classification algorithms can be limited on imbalanced datasets. In recent years, the imbalanced data learning problem has drawn significant interest. In this work, we focus on designing modifications to neural network, in order to appropriately tackle the problem of multiclass imbalance. We propose a hybrid method that combines two ideas: diverse random subspace ensemble learning with evolutionary search, to improve the performance of neural network on multiclass imbalanced data. An evolutionary search technique is utilized to optimize the misclassification cost under the guidance of imbalanced data measures. Moreover, the diverse random subspace ensemble employs the minimum overlapping mechanism to provide diversity so as to improve the performance of the learning and optimization of neural network. We have demonstrated experimentally using UCI datasets that our approach can achieve better result than state-of-the-art methods for imbalanced data.

Traditional classification algorithms can be limited in their performance on imbalanced datasets. In recent years, the imbalanced data learning problem has drawn significant interest. In this work, we focus on designing modifications to neural network, in order to appropriately tackle the problem of multiclass imbalance. We propose a method that combines two ideas: diverse random subspace ensemble learning with evolutionary search, to improve the performance of neural network on multiclass imbalanced data. An evolutionary search technique is utilized to optimize the misclassification cost under the guidance of imbalanced data measures. Moreover, the diverse random subspace ensemble employs the minimum overlapping mechanism to provide diversity so as to improve the performance of the learning and optimization of neural network. Furthermore, the ensemble framework can determine the optimal amount of non-redundant components automatically. We have demonstrated experimentally using UCI datasets that our approach can achieve significantly better result than state-of-the-art methods for imbalanced data.

Class imbalance is one of the challenging problems for machine learning in many real-world applications. Many methods have been proposed to address and attempt to solve the problem, including re-sampling and cost-sensitive learning. However, the existing methods have room for improvement since the potentially optimal values of the factors associated with best performance are unknown. Moreover most methods only focus on the binary class imbalance problem, thus there is no efficient solution in multi-class imbalanced learning. This paper presents an effective wrapper framework incorporating the evaluation measure into the objective function of cost sensitive learning as well as re-sampling directly, so as to improve the original methods through optimizing factors influencing the performance on the imbalanced data classification. Comprehensive experimental results on various standard benchmark datasets with different ratios of imbalance show that the influence of optimizing parameters on the solutions for learning imbalanced data is critical, and demonstrate the effectiveness of measure-optimized scheme on the imbalanced data learning.

Grouping data points is one of the fundamental tasks in data mining, which is commonly known as clustering if data points are described by attributes. When dealing with interrelated data that does not have any attributes and is represented in the form of nodes and their relationships, this task is also referred to as community mining. There has been a considerable number of approaches proposed in recent years for mining communities in a given network. But little work has been done on how to evaluate community mining results. The common practice is to use an agreement measure to compare the mining result against a ground truth, however, the ground truth is not known in most of the real world applications. In this paper, we investigate relative clustering quality measures defined for evaluation of clustering data points with attributes and propose proper adaptations to make them applicable in the context of social networks. Not only these relative criteria could be used as metrics for evaluating quality of the groupings but also they could be used as objectives for designing new community mining algorithms.

It is well known that the training of medical students is a long and arduous process. Students master many areas of knowledge in a relatively short amount of time in order to become experts in their chosen field. The Socratic Method used in the latter stages of medical education, where a physician directly monitors a group of students, is inherently restrictive due to the limited number of cases and length of the students’ rotations. Innovative Intelligent Tutoring techniques offer a solution to this problem. This paper outlines the overall structure and design of Shufti, an Intelligent Tutoring System (ITS) focused on mammography and medical imaging. Shufti's aim is to provide medical students with an improved learning environment, exposing them to a broad range of examples supported by customized feedback and hints driven by an adaptive Reinforcement Learning system and Clustering Techniques.

We present our COMmunity Boundary (COMB) and COMmunity Circles (COMC) network layout algorithms that focus on revealing the structure of discovered communities and the relationships between these communities. We believe this information is vital when developing new community mining algorithms as it allows the viewer to more quickly assess the quality of a mining result without appealing to large tables of statistics. To implement our algorithms we have introduced numerous modifications to the existing Fruchterman-Reingold layout, including support for multi-sized vertices, removal of the bounding frame, introduction of circular bounding boxes, and a novel slotting system. Our evaluation argues that both COMB and COMC outperform existing alternatives in their ability to reveal community structure and emphasize inter-community relations.

Top-k ranking for an uncertain database is to rank tuples in it so that the best k of them can be determined. The problem has been formalized under the unified approach based on parameterized ranking functions (PRFs) and the possible world semantics. Given a PRF, one can always compute the ranking function values of all the tuples to determine the top-k tuples, which is a formidable task for large databases. In this paper, we present a general approach to pruning for the framework based on PRFs. We show a mathematical manipulation of possible worlds which reveals key insights in the part of computation that may be pruned and how to achieve it in a systematic fashion. This leads to concrete pruning methods for a wide range of ranking functions. We show experimentally the effectiveness of our approach.

This paper describes a novel and fast placement algorithm for FPGA design space (e.g., area, power or reliability) exploration. The proposed algorithm generates the placement based on the topological similarity between two configurations (netlists) in the design space. Thus, it utilizes the sharing of reusable information during the design space exploration and avoids the time-consuming placement computation like VPR. Tested on logic-level and algorithm-level design space exploration cases, our similarity-based placement accurately depicts the “shape” of a design space and pinpoints the designs which are of most interest to IC designers. Moreover, a turbo version of circuit similarity-based placement performs an average of 30x (up to 100x) faster than VPR's while still achieving comparable placement results.

There has been much recent research on identifying global community structure in networks. However, most existing approaches require complete information of the graph in question, which is impractical for some networks, e.g. the World Wide Web (WWW). Algorithms for local community detection have been proposed but their results usually contain many outliers. In this paper, we propose a new measure of local community structure, coupled with a two-phase algorithm that extracts all possible candidates first, and then optimizes the community hierarchy. We compare our results with previous methods on real world networks such as the co-purchase network from Amazon. Experimental results verify the feasibility and effectiveness of our approach.

Communities in social networks may overlap, with some hub nodes belonging to multiple communities. They may also have outliers, which are nodes that belong to no community. The criterion to locate hubs or outliers is network dependent. Previous methods usually require this information as input parameters, e.g., an expected number of communities, with no intuition or assistance. Here we present a visual data mining approach, which first helps the user to make appropriate parameter selections by observing initial data visualizations, and then finds and extracts overlapping community structures from the network. Experimental results verify the scalability and accuracy of our approach on real network data and show its advantages over previous methods.

This paper introduces a new extension of outlier detection approaches and a new concept, class separation through variance. We show that accumulating information about the outlierness of points in multiple subspaces leads to a ranking in which classes with differing variance naturally tend to separate. Exploiting this leads to a highly effective and efficient unsupervised class separation approach, especially useful in the difficult case of heavily overlapping distributions. Unlike typical outlier detection algorithms, this method can be applied beyond the `rare classes' case with great success. Two novel algorithms that implement this approach are provided. Additionally, experiments show that the novel methods typically outperform other state-of-the-art outlier detection methods on high dimensional data such as Feature Bagging, SOE1, LOF, ORCA and Robust Mahalanobis Distance and competes even with the leading supervised classification methods.

Many datasets can be described in the form of graphs or networks where nodes in the graph represent entities and edges represent relationships between pairs of entities. A common property of these networks is their community structure, considered as clusters of densely connected groups of vertices, with only sparser connections between groups. The identification of such communities relies on some notion of clustering or density measure. which defines the communities that can be found. However, previous community detection methods usually apply the same structural measure on all kinds of networks, despite their distinct dissimilar features. In this paper, we present a new community mining measure, Max-Min Modularity, which considers both connected pairs and criteria defined by domain experts in finding communities, and then specify a hierarchical clustering algorithm to detect communities in networks. When applied to real world networks for which the community structures are already known, our method shows improvement over previous algorithms. In addition, when applied to randomly generated networks for which we only have approximate information about communities, it gives promising results which shows the algorithm's robustness against noise.

Group work is widespread in education. The growing use of online tools supporting group work generates huge amounts of data. We aim to exploit this data to support mirroring: presenting useful high-level views of information about the group, together with desired patterns characterizing the behavior of strong groups. The goal is to enable the groups and their facilitators to see relevant aspects of the group's operation and provide feedback if these are more likely to be associated with positive or negative outcomes and indicate where the problems are. We explore how useful mirror information can be extracted via a theory-driven approach and a range of clustering and sequential pattern mining. The context is a senior software development project where students use the collaboration tool TRAC. We extract patterns distinguishing the better from the weaker groups and get insights in the success factors. The results point to the importance of leadership and group interaction, and give promising indications if they are occurring. Patterns indicating good individual practices were also identified. We found that some key measures can be mined from early data. The results are promising for advising groups at the start and early identification of effective and poor practices, in time for remediation.

The classification of an unknown item based on a training data set is a key data mining task. An important part of this process that is often overlooked is the user's comprehension of the classifier and the results it produces. Associative classifiers begin to address this issue by using sets of simple rules to classify items. However, the size of these rule sets can be an obstacle to understandability. In this work, we present an interactive visualization system that allows the user to visualize various aspects of the classifier's decision process. This system shows the rules that are relevant to the classification of an item, the ways in which the item's characteristics relate to these rules, and connections between the item and the classifier's training data set. The system also contains a speculation component, which allows the user to modify rules within the classifier, and see the impact of these changes. Thus, this component allows the user to contribute domain expertise to the classification process, consequently improving the accuracy of the classifier.

Effectively organizing web search results into clusters is important to facilitate quick user navigation to relevant documents. Previous methods may rely on a training process and do not provide a measure for whether page clustering is actually required. In this paper, we reformalize the clustering problem as a word sense discovery problem. Given a query and a list of result pages, our unsupervised method detects word sense communities in the extracted keyword network. The documents are assigned to several refined word sense communities to form clusters. We use the modularity score of the discovered keyword community structure to measure page clustering necessity. Experimental results verify our method's feasibility and effectiveness.

The need to identify significant differences between contrasting groups or classes is ubiquitous and thus was the focus of many statisticians and data miners. Contrast sets, conjunctions of attribute-value pairs significantly more frequent in one group than another, were proposed to describe such differences, which lead to the introduction of a new data mining technique - contrast-set mining. A number of attempts have been made in this regard by various authors; however, no clear picture seems to have emerged. In this paper, we try to address the problem of finding meaningful contrast sets by using association rule based analysis. We present the results for our experiments for interesting contrast sets and compare these results with those obtained from the well-known algorithm for contrast sets-STUCCO.

Feature space conversion for classifiers is the process by which the data that is to be fed into the classifier is transformed from one form to another. The motivation behind doing this is to enhance the "discriminative power" of the data together with preserving its "information content". In this paper, a new method of feature space conversion is explored, wherein "enrichment" of the feature space is carried out by the augmentation of the existing features with new "implicit" features. The modus operandi involves generation of association rules in one case and closed frequent patterns in another and the extraction of the new features from these. This new feature space is first made use of independently to feed the classifier and then it is used in unison with the original feature space. The effectiveness of these methods is subsequently verified experimentally and expressed in terms of the classification accuracy achieved by the classifier.

As the volume of digitally accessible information grows, there is increasing pressure on the development of data visualization methods to enable humans to interpret that data. We provide a description of our WebViz system, as a tool to visualize both the structure and usage of web sites. We illustrate the use of our visualization paradigm by introducing polygonal graphs layered on top of our adaptation of radial disk trees. In our system, the structure of a web segment is rendered as a radial tree, and usage data can be extracted and layered as polygonal graphs. By interactively creating and adjusting these layers, a user can develop real time insight into the data. We present the system, show the idea of interactive visual operators, and provide some examples that help show the value of the specific visualization techniques, as well as the interactive use of those techniques.

Discovering web navigational trends and understanding data mining results is undeniably advantageous to web designers and web-based application builders. It is also desirable to interactively investigate web access data and patterns, to allows ad-hoc discovery and examination of patterns that are not apriori known. Visualizing the usage data in the context of the web site structure is of major importance, as it puts web access requests and their connectivity in perspective. Various visualization tools have been developed for this task, but often fail to provide visual data mining functionalities to generate new patterns. Here we present our visual data mining system, WebViz, which allows interactive investigation of web usage data within their structure context, as well as ad-hoc knowledge pattern discovery on web navigational behaviour.

While the sharing of data is known to be beneficial in data mining applications and widely acknowledged as advantageous in business, this information sharing can become controversial and thwarted by privacy regulations and other privacy concerns. Data clustering for instance could be more accurate if more information is available, hence the data sharing. Any solution needs to balance the clustering requirements and the privacy issues. Rather than simply hindering data owners from sharing information for data analysis, a solution could be designed to meet privacy requirements and guarantee valid data clustering results. To achieve this dual goal, this article introduces a method for privacy-preserving clustering called dimensionality reduction-based transformation (DRBT). This method relies on the intuition behind random projection to protect the underlying attribute values subjected to cluster analysis. It is shown analytically and empirically that transforming a dataset using DRBT, a data owner can achieve privacy preservation and get accurate clustering with little overhead of communication cost. Such a method presents the following advantages: it is independent of distance-based clustering algorithms, it has a sound mathematical foundation, and it does not require CPU-intensive operations.

Association rule-based classifiers have recently emerged as competitive classification systems. However, there are still deficiencies that hinder their performance. One deficiency is the use of rules in the classification stage. Current systems assign classes to new objects based on the best rule applied or on some predefined scoring of multiple rules. In this paper we propose a new technique where the system automatically learns how to use the rules. We achieve this by developing a two-stage classification model. First, we use association rule mining to discover classification rules. Second, we employ another learning algorithm to learn how to use these rules in the prediction process. Our two-stage approach outperforms C4.5 and RIPPER on the UCI datasets in our study, and outperforms other rule- learning methods on more than half the datasets. The versatility of our method is also demonstrated by applying it to text classification, where it equals the performance of the best known systems for this task, SVMs.

A bottleneck to detecting distance and density based outliers is that a nearest-neighbor search is required for each of the data points, resulting in a quadratic number of pairwise distance evaluations. In this paper, we propose a new method that uses the relative degree of density with respect to a fixed set of reference points to approximate the degree of density defined in terms of nearest neighbors of a data point. The running time of our algorithm based on this approximation is 0(Rn log n) where n is the size of dataset and R is the number of reference points. Candidate outliers are ranked based on the outlier score assigned to each data point. Theoretical analysis and empirical studies show that our method is effective, efficient, and highly scalable to very large datasets.

Benchmarking technical solutions is as important as the solutions themselves. Yet many fields still lack any type of rigorous evaluation. Performance benchmarking has always been an important issue in databases and has played a significant role in the development, deployment and adoption of technologies. To help assessing the myriad algorithms for frequent itemset mining, we built an open framework and testbed to analytically study the performance of different algorithms and their implementations, and contrast their achievements given different data characteristics, different conditions, and different types of patterns to discover and their constraints. This facilitates reporting consistent and reproducible performance results using known conditions.

Mining for frequent item sets can generate an overwhelming number of patterns, often exceeding the size of the original transactional database. One way to deal with this issue is to set filters and interestingness measures. Others advocate the use of constraints to apply to the patterns, either on the form of the patterns or on descriptors of the items in the patterns. However, typically the filtering of patterns based on these constraints is done as a post-processing phase. Filtering the patterns post-mining adds a significant overhead, still suffers from the sheer size of the pattern set and loses the opportunity to exploit those constraints. In this paper we propose an approach that allows the efficient mining of frequent item sets patterns, while pushing simultaneously both monotone and anti-monotone constraints during and at different strategic stages of the mining process. Our implementation shows a significant improvement when considering the constraints early and a better performance over Dualminer which also considers both types of constraints.

Regardless of the frequent patterns to discover, either the full frequent patterns or the condensed ones, either closed or maximal, the strategy always includes the traversal of the lattice of candidate patterns. We study the existing depth versus breadth traversal approaches for generating candidate patterns and propose in this paper a new traversal approach that jumps in the search space among only promising nodes. Our leaping approach avoids nodes that would not participate in the answer set and reduce drastically the number of candidate patterns. We use this approach to efficiently pinpoint maximal patterns at the border of the frequent patterns in the lattice and collect enough information in the process to generate all subsequent patterns.

Web site structures are complex to analyze. Cross-referencing the web structure with navigational behaviour adds to the complexity of the analysis. However, this convoluted analysis is necessary to discover useful patterns and understand the navigational behaviour of web site visitors, whether to improve web site structures, provide intelligent on-line tools or offer support to human decision makers. Moreover, interactive investigation of web access logs is often desired since it allows ad hoc discovery and examination of patterns not a priori known. Various visualization tools have been provided for this task but they often lack the functionality to conveniently generate new patterns. In this paper we propose a visualization tool to visualize web graphs, representations of web structure overlaid with information and pattern tiers. We also propose a web graph algebra to manipulate and combine web graphs and their layers in order to discover new patterns in an ad hoc manner.

We address the problem of protecting some sensitive knowledge in transactional databases. The challenge is on protecting actionable knowledge for strategic decisions, but at the same time not losing the great benefit of association rule mining. To accomplish that, we introduce a new, efficient one-scan algorithm that meets privacy protection and accuracy in association rule mining, without putting at risk the effectiveness of the data mining per se.

An image mosaic is an image made up of many other images. In this paper we investigate the automatic generation of such image mosaics, using content-based image retrieval as the underlying framework. Our first contribution is to describe and evaluate a few parameters that control the quality of the mosaic image. Our second contribution is the proposal of an (automatic) measure to assess the quality of the resulting images. Several examples of mosaics built in the context of this research can be found at http://db.cs.ualberta.ca/mosaicture.

In this paper, we propose a novel data structure called CATS Tree. CATS Tree extends the idea of FPTree to improve storage compression and allow frequent pattern mining without generation of candidate item sets. The proposed algorithms enable frequent pattern mining with different supports without rebuilding the tree structure. Furthermore, the algorithms allow mining with a single pass over the database as well as efficient insertion or deletion of transactions at any time.

The discovery of association rules from large databases has proven beneficial for companies since such rules can be very effective in revealing actionable knowledge that leads to strategic decisions. In tandem with this benefit, association rule mining can also pose a threat to privacy protection. The main problem is that from non-sensitive information or unclassified data, one is able to infer sensitive information, including personal information, facts, or even patterns that are not supposed to be disclosed. This scenario reveals a pressing need for techniques that ensure privacy protection, while facilitating proper information accuracy and mining. In this paper, we introduce new algorithms for balancing privacy and knowledge discovery in association rule mining. We show that our algorithms require only two scans, regardless of the database size and the number of restrictive association rules that must be protected. Our performance study compares the effectiveness and scalability of the proposed algorithms and analyzes the fraction of association rules, which are preserved after sanitizing a database. We also report the main results of our performance evaluation and discuss some open research issues.

Existing association rule mining algorithms suffer from many problems when mining massive transactional datasets. One major problem is the high memory dependency: either the gigantic data structure built is assumed to fit in main memory, or the recursive mining process is too voracious in memory resources. Another major impediment is the repetitive and interactive nature of any knowledge discovery process. To tune parameters, many runs of the same algorithms are necessary leading to the building of these huge data structures time and again. This paper proposes a new disk-based association rule mining algorithm called Inverted Matrix, which achieves its efficiency by applying three new ideas. First, transactional data is converted into a new database layout called Inverted Matrix that prevents multiple scanning of the database during the mining phase, in which finding frequent patterns could be achieved in less than a full scan with random access. Second, for each frequent item, a relatively small independent tree is built summarizing co-occurrences. Finally, a simple and non-recursive mining process reduces the memory requirements as minimum candidacy generation and counting is needed. Experimental studies reveal that our Inverted Matrix approach outperform FP-Tree especially in mining very large transactional databases with a very large number of unique items. Our random access disk-based approach is particularly advantageous in a repetitive and interactive setting.

A recommender system in an e-learning context is a software agent that tries to "intelligently" recommend actions to a learner based on the actions of previous learners. This recommendation could be an on-line activity such as doing an exercise, reading posted messages on a conferencing system, or running an on-line simulation, or could be simply a web resource. These recommendation systems have been tried in e-commerce to entice purchasing of goods, but haven't been tried in e-learning. This paper suggests the use of web mining techniques to build such an agent that could recommend on-line learning activities or shortcuts in a course web site based on learners' access history to improve course material navigation as well as assist the online learning process. These techniques are considered integrated web mining as opposed to off-line web mining used by expert users to discover on-line access patterns.

A good text classifier is a classifier that efficiently categorizes large sets of text documents in a reasonable time frame and with an acceptable accuracy, and that provides classification rules that are human readable for possible fine-tuning. If the training of the classifier is also quick, this could become in some application domains a good asset for the classifier. Many techniques and algorithms for automatic text categorization have been devised. According to published literature, some are more accurate than others, and some provide more interpretable classification models than others. However, none can combine all the beneficial properties enumerated above. In this paper we present a novel approach for automatic text categorization that borrows from market basket analysis techniques using association rule mining in the data-mining field. We focus on two major problems: (1) finding the best term association rules in a textual database by generating and pruning; and (2) using the rules to build a text classifier. Our text categorization method proves to be efficient and effective, and experiments on well-known collections show that the classifier performs well. In addition, training as well as classification are both fast and the generated rules are human readable.

Clustering is the problem of grouping data based on similarity and consists of maximizing the intra-group similarity while minimizing the inter-group similarity. The problem Of clustering data sets is also known as unsupervised classification, since no class labels are given. However, all existing clustering algorithms require some parameters to steer the clustering process, such as the famous k for the number of expected clusters, which constitutes a supervision of a sort. We present in this paper a new, efficient, fast and scalable clustering algorithm that clusters over a range of resolutions and finds a potential optimum clustering without requiring any parameter input. Our experiments show that our algorithm outperforms most existing clustering algorithms in quality and speed for large data sets.

Clustering spatial data is a well-known problem that has been extensively studied to find hidden patterns or meaningful sub-groups and has many applications such as satellite imagery, geographic information systems, medical image analysis, etc. Although many methods have been proposed in the literature, very few have considered constraints such that physical obstacles and bridges linking clusters may have significant consequences on the effectiveness of the clustering. Taking into account these constraints during the clustering process is costly, and the effective modeling of the constraints is of paramount importance for good performance. In this paper we define the clustering problem in the presence of constraints - obstacles and crossings - and investigate its efficiency and effectiveness for large databases. In addition, we introduce a new approach to model these constraints to prune the search space and reduce the number of polygons to test during clustering. The algorithm DBCluC we present detects clusters of arbitrary shape and is insensitive to noise and the input order Its average running complexity is O(NlogN) where N is the number of data objects.

Clustering spatial data is a well-known problem that has been extensively studied. Grouping similar data in large 2-dimensional spaces to find hidden patterns or meaningful sub-groups has many applications such as satellite imagery, geographic information systems, medical image analysis, marketing, computer visions, etc. Although many methods have been proposed in the literature, very few have considered physical obstacles that may have significant consequences on the effectiveness of the clustering. Taking into account these constraints during the clustering process is costly and the modeling of the constraints is paramount for good performance. In this paper, we investigate the problem of clustering in the presence of constraints such as physical obstacles and introduce a new approach to model these constraints using polygons. We also propose a strategy to prune the search space and reduce the number of polygons to test during clustering. We devise a density-based clustering algorithm, DBCluC, which takes advantage of our constraint modeling to efficiently cluster data objects while considering all physical constraints. The algorithm can detect clusters of arbitrary shape and is insensitive to noise, the input order and the difficulty of constraints. Its average running complexity is O(NlogN) where N is the number of data points.

The accessibility of the World Wide Web and the ease of use of the tools to browse the resources on the Web have made this technology extremely popular and the means of choice for distance education. Many sophisticated Web-based learning environments have been developed and are in use around the world. Educators using these environments and tools, however, have very little support to evaluate learners' activities and discriminate between different learner's online behaviours. The authors exploit the existence of Web access logs and advanced data mining techniques to extract useful patterns that can help educators and Web masters evaluate and interpret online course activities in order to assess the learning process, track student actions and measure Web course structure effectiveness.

In this paper we introduce a new parallel algorithm MLFPT (multiple local frequent pattern tree) for parallel mining of frequent patterns, based on FP-growth mining, that uses only two full I/O scans of the database, eliminating the need for generating candidate items, and distributing the work fairly among processors. We have devised partitioning strategies at different stages of the mining process to achieve near optimal balancing between processors. We have successfully tested our algorithm on datasets larger than 50 million transactions.

Despite the overwhelming amounts of multimedia data recently generated and the significance of such data, very few people have systematically investigated multimedia data mining. With our previous studies on content-based retrieval of visual artifacts, we study in this paper the methods for mining content-based associations with recurrent items and with spatial relationships from large visual data repositories. A progressive resolution refinement approach is proposed in which frequent item-sets at rough resolution levels are mined, and progressively, finer resolutions are mined only on the candidate frequent items-sets derived from mining rough resolution levels. Such a multi-resolution mining strategy substantially reduces the overall data mining cost without loss of the quality and completeness of the results.

No abstract available.

Multimedia data mining is the mining of high-level multimedia information and knowledge from large multimedia databases. A multimedia data mining system prototype, MultiMediaMiner, has been designed and developed. It includes the construction of a multimedia data cube which facilitates multiple dimensional analysis of multimedia data, primarily based on visual content, and the mining of multiple kinds of knowledge, including summarization, comparison, classification, association, and clustering.

Based on our years-of-research, a data mining system, DB-Miner, has been developed for interactive mining of multiple-level knowledge in large relational databases. The system implements a wide spectrum of data mining functions, including generalization, characterization, association, classification, and prediction. By incorporation of several interesting data mining techniques, including attribute-oriented induction, progressive deepening for mining multiple-level rules, and meta-rule guided knowledge mining, the system provides a user-friendly, interactive data mining environment with good performance.