Recent advances in large language models have strengthened Text2SQL systems that translate natural language questions into database queries. A persistent deployment challenge is to assess a newly trained Text2SQL system on an unseen and unlabeled dataset when no verified answers are available. This situation arises frequently because database content and structure evolve, privacy policies slow manual review, and carefully written SQL labels are costly and time-consuming. Without timely evaluation, organizations cannot approve releases or detect failures early. FusionSQL addresses this gap by working with any Text2SQL models and estimating accuracy without reference labels, allowing teams to measure quality on unseen and unlabeled datasets. It analyzes patterns in the system's own outputs to characterize how the target dataset differs from the material used during training. FusionSQL supports pre-release checks, continuous monitoring of new databases, and detection of quality decline. Experiments across diverse application settings and question types show that FusionSQL closely follows actual accuracy and reliably signals emerging issues. Our code is available at https://github.com/phkhanhtrinh23/FusionSQL.

The rapid advancement of artificial intelligence (AI) systems such as large language models (LLMs) and multimodal learning frameworks has transformed digital content creation and manipulation. Traditional visual editing tools require significant expertise, limiting accessibility. Recent strides in instruction-guided editing have enabled intuitive interaction with visual content, using natural language as a bridge between user intent and complex editing operations. This survey reviews how AI-implemented instruction-guided image editing models – including approaches rooted in generative adversarial networks and diffusion models – empower users to achieve precise visual modifications without deep technical knowledge. By synthesizing over 100 publications, we examine multimodal integration for fine-grained content control, compare existing literature, and highlight how these AI applications support creative visual storytelling, design workflows, and multimedia production. We also identify key challenges to stimulate further research. Interested readers are encouraged to access our repository at https://github.com/tamlhp/awesome-instructional-editing .

• Multi-view contrastive learning overcomes data sparsity • Temporal regularisation stabilises user preferences across sequences • Popularity-aware defence mitigates promotion and camouflage attacks • Achieves stable performance within ± 1% even with 80% malicious clients • Maintain 90% of original user accuracy even when cold-start Federated sequential recommendation (FedSeqRec) allows many user devices to train a shared recommender without sending raw interaction histories to a central server, which is important for privacy. However, existing FedSeqRec methods still suffer from two key limitations: (1) most users have very short or sparse histories, especially when they only show interest in a few items for a short period, leaving the model with too little data to understand their preferences; and (2) in a sequential setting it is normal for interests to change suddenly, but the model may misinterpret these abrupt changes as anomalies. In this paper, we propose FORTRESS , a F ederated c O ntrastive R obus T RE commender for S equential S ystems, designed to address these limitations. To tackle the first issue, FORTRESS generates augmented versions of local interaction sequences on each client, so that the model can observe more plausible behaviour patterns and learn user preferences more reliably even when histories are short or sparse. This extra flexibility also gives adversaries more room to manipulate the training signal, so we complement it with a popularity-aware server-side regularizer that discourages rare or suspicious items from drifting into the same embedding clusters as genuinely popular items. To tackle the second issue, we introduce a temporal regularization term that discourages abrupt changes in user representations across adjacent subsequences, allowing the model to adapt to short-term interest shifts while still preserving stable long-term tastes. Experiments on three real-world datasets show that FORTRESS improves recommendation accuracy for sparse and cold-start users and substantially reduces the success of strong model poisoning attacks compared with competitive centralized and federated baselines.

Medical Large Multi-modal Models (LMMs) have demonstrated remarkable capabilities in medical data interpretation. However, these models frequently generate hallucinations contradicting source evidence, particularly due to inadequate localization reasoning. This work reveals a critical limitation in current medical LMMs: instead of analyzing relevant pathological regions, they often rely on linguistic patterns or attend to irrelevant image areas when responding to disease-related queries. To address this, we introduce HEAL-MedVQA (Hallucination Evaluation via Localization MedVQA), a comprehensive benchmark designed to evaluate LMMs' localization abilities and hallucination robustness. HEAL-MedVQA features (i) two innovative evaluation protocols to assess visual and textual shortcut learning, and (ii) a dataset of 67K VQA pairs, with doctor-annotated anatomical segmentation masks for pathological regions. To improve visual reasoning, we propose the Localize-before-Answer (LobA) framework, which trains LMMs to localize target regions of interest and self-prompt to emphasize segmented pathological areas, generating grounded and reliable answers. Experimental results demonstrate that our approach significantly outperforms state-of-the-art biomedical LMMs on the challenging HEAL-MedVQA benchmark, advancing robustness in medical VQA.

Vision-Language Foundation Models (VLMs), trained on large-scale multimodal datasets, have driven significant advances in Artificial Intelligence (AI) by enabling rich cross-modal reasoning. Despite their success in general domains, applying these models to medical imaging remains challenging due to the limited availability of diverse imaging modalities and multilingual clinical data. Most existing medical VLMs are trained on a subset of imaging modalities and focus primarily on high-resource languages, thus limiting their generalizability and clinical utility. To address these limitations, we introduce a novel Vietnamese-language multimodal medical dataset consisting of 2,757 whole-body PET/CT volumes from independent patients and their corresponding full-length clinical reports. This dataset is designed to fill two pressing gaps in medical AI development: (1) the lack of PET/CT imaging data in existing VLMs training corpora, which hinders the development of models capable of handling functional imaging tasks; and (2) the underrepresentation of low-resource languages, particularly the Vietnamese language, in medical vision-language research. To the best of our knowledge, this is the first dataset to provide comprehensive PET/CT-report pairs in Vietnamese. We further introduce a training framework to enhance VLMs' learning, including data augmentation and expert-validated test sets. We conduct comprehensive experiments benchmarking state-of-the-art VLMs on downstream tasks, including medical report generation and visual question answering. The experimental results show that incorporating our dataset significantly improves the performance of existing VLMs. However, despite these advancements, the models still underperform on clinically critical criteria, particularly the diagnosis of lung cancer, indicating substantial room for future improvement. We believe this dataset and benchmark will serve as a pivotal step in advancing the development of more robust VLMs for medical imaging, particularly in low-resource languages, and improving their clinical relevance in Vietnamese healthcare.

Abstract The evolution of edge computing has advanced the accessibility of E-health recommendation services, encompassing areas such as medical consultations, prescription guidance, and diagnostic assessments. Traditional methodologies predominantly utilize centralized recommendations, relying on servers to store client data and dispatch advice to users. However, these conventional approaches raise significant concerns regarding data privacy and often result in computational inefficiencies. E-health recommendation services, distinct from other recommendation domains, demand not only precise and swift analyses but also a stringent adherence to privacy safeguards, given the users’ reluctance to disclose their identities or health information. In response to these challenges, we explore a new paradigm called on-device recommendation tailored to E-health diagnostics, where diagnostic support (such as biomedical image diagnostics), is computed at the client level. We leverage the advances of federated learning to deploy deep learning models capable of delivering expert-level diagnostic suggestions on clients. However, existing federated learning frameworks often deploy a singular model across all edge devices, overlooking their heterogeneous computational capabilities. In this work, we propose an adaptive federated learning framework utilizing BlockNets, a modular design rooted in the layers of deep neural networks, for diagnostic recommendation across heterogeneous devices. Our framework offers the flexibility for users to adjust local model configurations according to their device’s computational power. To further handle the capacity skewness of edge devices, we develop a data-free knowledge distillation mechanism to ensure synchronized parameters of local models with the global model, enhancing the overall accuracy. Through comprehensive experiments across five real-world datasets, against six baseline models, within six experimental setups, and various data distribution scenarios, our architecture demonstrates unparalleled performance and robustness in terms of both accuracy and efficiency.

Abstract As the adoption of explainable AI (XAI) continues to expand, the urgency to address its privacy implications intensifies. Despite a growing corpus of research in AI privacy and explainability, there is little attention on privacy-preserving model explanations. This article presents the first thorough survey about privacy attacks on model explanations and their countermeasures. Our contribution to this field comprises a thorough analysis of research papers with a connected taxonomy that facilitates the categorization of privacy attacks and countermeasures based on the targeted explanations. This work also includes an initial investigation into the causes of privacy leaks. Finally, we discuss unresolved issues and prospective research directions uncovered in our analysis. This survey aims to be a valuable resource for the research community and offers clear insights for those new to this domain. To support ongoing research, we have established an online resource repository, which will be continuously updated with new and relevant findings.

Today, computer systems hold large amounts of personal data. Yet while such an abundance of data allows breakthroughs in AI, and especially machine learning, its existence can be a threat to user privacy, and it can weaken the bonds of trust between humans and AI. Recent regulations now require that, on request, private information about a user must be removed both from computer systems and from machine learning models—this legislation is more colloquially called “the right to be forgotten.” While removing data from back-end databases should be straightforward, it is not sufficient in the AI context as machine learning models often “remember” the old data. Contemporary adversarial attacks on trained models have proven that we can learn whether an instance or an attribute belonged to the training data. This phenomenon calls for a new paradigm, namely machine unlearning , to make machine learning models forget about particular data. It turns out that recent works on machine unlearning have not been able to completely solve the problem due to the lack of common frameworks and resources. Therefore, this article aspires to present a comprehensive examination of machine unlearning’s concepts, designs, methods, and applications. Specifically, as a category collection of cutting-edge studies, the intention behind this article is to serve as a comprehensive resource for researchers and practitioners seeking an introduction to machine unlearning and its formulations, design criteria, removal requests, algorithms, and applications. In addition, we aim to highlight the key findings, current trends, and new research areas that have not yet featured the use of machine unlearning but could benefit greatly from it. We hope that this survey serves as a valuable resource for machine learning researchers and those seeking to innovate privacy technologies. Our resources are publicly available at https://github.com/tamlhp/awesome-machine-unlearning .

Modern recommender systems derive predictions from an interaction graph that links users and items. To this end, many of today's state-of-the-art systems use graph neural networks (GNNs) to learn effective representations of these graphs under the assumption of homophily, i.e., the idea that similar users will sit close to each other in the graph. However, recent studies have revealed that real-world recommendation graphs are often heterophilous, i.e., dissimilar users will also often sit close to each other. One of the reasons for this heterophilia is shilling attacks that obscure the inherent characteristics of the graph and make the derived recommendations less accurate as a consequence. Hence, to cope with low homophily in recommender systems, we propose a recommendation model called PGT4Rec that is based on a Partitioned Graph Transformer. The model integrates label information into the learning process, which allows discriminative neighbourhoods of users to be generated. As such, the framework can both detect shilling attacks and predict user ratings for items. Extensive experiments on real and synthetic datasets show PGT4Rec as not only providing superior performance in these two tasks but also significant robustness to a range of adversarial conditions.

Recommendation systems play a crucial role in providing web-based suggestion utilities by leveraging user behavior, preferences, and interests. In the context of privacy concerns and the proliferation of handheld devices, federated recommender systems have emerged as a promising solution. These systems allow each client to train a local model and exchange only the model updates with a central server, thus preserving data privacy. However, certain use cases necessitate the deduction of contributions from specific clients, a process known as “unlearning.” Existing machine unlearning methods are designed for centralized settings and do not cater to the collaborative nature of recommendation systems, thereby overlooking their unique characteristics. This article proposes CFRU, a novel federated recommendation unlearning model that enables efficient and certified removal of target clients from the global model. Instead of retraining the model, our approach rolls back and eliminates the historical updates associated with the target client. To efficiently store the learning process’s historical updates, we propose sampling strategies that reduce the number of historical updates, retaining only the most significant ones. Furthermore, we analyze the potential bias introduced by the removal of target clients’ updates at each training round and establish an estimation using the Lipschitz condition. Leveraging this estimation, we propose an efficient iterative scheme to accumulate the bias across all rounds, compensating for the removed updates from the global model and recovering its utility without requiring post-training steps. Extensive experiments conducted on two real-world datasets, incorporating two poison attack scenarios, have shown that our unlearning technique can achieve a model quality that is 99.3% equivalent to retraining the model from scratch while performing up to 1,000 times faster.

Recent advancements in recommender systems have focused on integrating knowledge graphs (KGs) to leverage their auxiliary information. The core idea of KG-enhanced recommenders is to incorporate rich semantic information for more accurate recommendations. However, two main challenges persist: i) Neglecting complex higher-order interactions in the KG-based user-item network, potentially leading to sub-optimal recommendations, and ii) Dealing with the heterogeneous modalities of input sources, such as user-item bipartite graphs and KGs, which may introduce noise and inaccuracies. To address these issues, we present a novel Knowledge-enhanced Heterogeneous Hypergraph Recommender System (KHGRec). KHGRec captures group-wise characteristics of both the interaction network and the KG, modeling complex connections in the KG. Using a collaborative knowledge heterogeneous hypergraph (CKHG), it employs two hypergraph encoders to model group-wise interdependencies and ensure explainability. Additionally, it fuses signals from the input graphs with cross-view self-supervised learning and attention mechanisms. Extensive experiments on four real-world datasets show our model's superiority over various state-of-the-art baselines, with an average 5.18% relative improvement. Additional tests on noise resilience, missing data, and cold-start problems demonstrate the robustness of our KHGRec framework. Our model and evaluation datasets are publicly available at https://github.com/viethungvu1998/KHGRec.

The propagation of rumours on social media poses an important threat to societies, so that various techniques for graph-based rumour detection have been proposed recently. Existing works, however, are based on homophilic graphs: entities that are connected to each other often have the same label. However, recent studies found that heterophily is more common in real-world social networks, i.e., entities with different labels are also often linked to each other due to ‘innocent’ retweets or camouflage behaviours by malicious users. Especially, the heterophily problem is even more challenging in multi-modal social graphs, in which neighbouring entities might differ in terms of both labels and modalities. To cope with multi-modal homophily in graph-based rumour detection, we propose a Portable Graph Transformer-based Rumour Detection model (PHAROS) with novel multi-modal homophily measures. It integrates label information in the learning process, which enables us to generate discriminative neighbourhoods of entities. Our model can handle multiple modalities (a natural characteristic of social graphs) and is portable to be combined with existing graph-based models. Extensive experiments on real and synthetic data show the superiority, efficiency, robustness, and portability of PHAROS and its heterophily resilience.

Maximal frequent subgraph mining (MFSM) is the task of mining only maximal frequent subgraphs, i.e., subgraphs that are not a part of other frequent subgraphs. Although many intelligent systems require MFSM, MFSM is challenging compared to frequent subgraph mining (FSM), as maximal frequent subgraphs lie in the middle of graph lattice, and FSM algorithms must explore an exponential space and an NP-hard subroutine of frequency counting. Different from prior research, which primarily focused on optimal solutions, we introduce pmMine, a progressive graph neural framework designed for MFSM in a single large graph to attain an approximate solution. The framework combines isomorphic graph embedding, non-parametric partitioning, and an efficiently top-down pattern searching strategy. The critical insight that makes pmMine work is to define the concepts of rooted subgraph and isomorphic graph embedding, in which the costly isomorphism subroutine can be efficiently performed using similarity estimation in embedding space. In addition, pmMine returns the patterns identified during the mining process in a progressive manner. We validate the efficiency and effectiveness of our technique through extensive experiments on a variety of datasets spanning various domains.

Speech emotion recognition (SER) has been a popular research topic in human-computer interaction (HCI). As edge devices are rapidly springing up, applying SER to edge devices is promising for a huge number of HCI applications. Although deep learning has been investigated to improve the performance of SER by training complex models, the memory space and computational capability of edge devices represents a constraint for embedding deep learning models. We propose a neural structured learning (NSL) framework through building synthesized graphs. An SER model is trained on a source dataset and used to build graphs on a target dataset. A relatively lightweight model is then trained with the speech samples and graphs together as the input. Our experiments demonstrate that training a lightweight SER model on the target dataset with speech samples and graphs can not only produce small SER models, but also enhance the model performance compared to models with speech samples only and those using classic transfer learning strategies.

Speech emotion recognition (SER) is the task of recognising humans’ emotional states from speech. SER is extremely prevalent in helping dialogue systems to truly understand our emotions and become a trustworthy human conversational partner. Due to the lengthy nature of speech, SER also suffers from the lack of abundant labelled data for powerful models like deep neural networks. Pre-trained complex models on large-scale speech datasets have been successfully applied to SER via transfer learning. However, fine-tuning complex models still requires large memory space and results in low inference efficiency. In this paper, we argue achieving a fast yet effective SER is possible with self-distillation, a method of simultaneously fine-tuning a pretrained model and training shallower versions of itself. The benefits of our self-distillation framework are threefold: (1) the adoption of self-distillation method upon the acoustic modality breaks through the limited ground-truth of speech data, and outperforms the existing models’ performance on an SER dataset; (2) executing powerful models at different depths can achieve adaptive accuracy-efficiency trade-offs on resource-limited edge devices; (3) a new fine-tuning process rather than training from scratch for self-distillation leads to faster learning time and the state-of-the-art accuracy on data with small quantities of label information.

Recent information retrieval (IR) systems answer a multi-modal query by considering it as a set of separate uni-modal queries. However, depending on the chosen operationalisation, such an approach is inefficient or ineffective. It either requires multiple passes over the data or leads to inaccuracies since the relations between data modalities are neglected in the relevance assessment. To mitigate these challenges, we present an IR system that has been designed to answer genuine multi-modal queries. It relies on a heterogeneous network embedding, so that features from diverse modalities can be incorporated when representing both, a query and the data over which it shall be evaluated. An experimental evaluation using diverse real-world and synthetic datasets illustrates that our approach returns twice the amount of relevant information compared to baseline techniques, while scaling to large multi-modal databases.

The goal of subgraph matching is to determine the presence of a particular query pattern within a large collection of data graphs. Despite being a hard problem, subgraph matching is essential in various disciplines, including bioinformatics, text matching, and graph retrieval. Although traditional approaches could provide exact solutions, their computations are known to be NP-complete, leading to an overwhelmingly querying latency. While recent neural-based approaches have been shown to improve the response time, the oversimplified assumption of the first-order network may neglect the generalisability of fully capturing patterns in varying sizes, causing the performance to drop significantly in datasets in various domains. To overcome these limitations, this paper proposes xDualSM, a dual matching neural network model with interleaved diffusion attention. Specifically, we first embed the structural information of graphs into different adjacency matrices, which explicitly capture the intra-graph and cross-graph structures between the query pattern and the target graph. Then, we introduce a dual matching network with interleaved diffusion attention to carefully capture intra-graph and cross-graph information while reducing computational complexity. Empirically, our proposed framework not only boosted the speed of subgraph matching more than 10x compared to the fastest baseline but also achieved significant improvements of 47.64% in Recall and 34.39% in F1-score compared to the state-of-the-art approximation approach on COX2 dataset. In addition, our results are comparable with exact methods.

Analysis of respiratory sounds is an area where deep neural networks (DNNs) may benefit clinicians and patients for diagnostic purposes due to their classification power. However, explaining the predictions made by DNNs remains a challenge. Currently, most explanation methods focus on post-hoc explanations, where a separate explanatory model is used to explain a trained DNN. Due to the complex nature of respiratory sound classification pipeline involving signal processing such as frequency analysis and wavelet analysis, post-hoc methods cannot uncover the underlying inference process of DNNs, highlighting the importance of designing DNNs with intrinsic interpretability. In this paper, we propose a self-explaining DNN for respiratory sound classification based on prototype learning. Our model explains its behavior by generating sample prototypes while attaching these prototypes to a layer inside the neural network. Furthermore, we design a scale-free interpretability mechanism, in which the model reaches its final decision by dissecting the input and looking for similarities between several parts of the input and the prototypes. The experimental findings on the largest public respiratory sound database demonstrate that our method achieves comparable, sometimes better, performance with the non-interpretable counterparts while offering state-of-the-art interpretability. The code will be released upon acceptance.

Advances in deep neural network (DNN) architectures have enabled new prediction techniques for stock market data. Unlike other multivariate time-series data, stock markets show two unique characteristics: (i) multi-order dynamics, as stock prices are affected by strong non-pairwise correlations (e.g., within the same industry); and (ii) internal dynamics, as each individual stock shows some particular behaviour. Recent DNN-based methods capture multi-order dynamics using hypergraphs, but rely on the Fourier basis in the convolution, which is both inefficient and ineffective. In addition, they largely ignore internal dynamics by adopting the same model for each stock, which implies a severe information loss.

The task of discovering equivalent entities in knowledge graphs (KGs), so‐called KG entity alignment, has drawn much attention to overcome the incompleteness problem of KGs. The majority of existing techniques learns the pointwise representations of entities in the Euclidean space with translation assumption and graph neural network approaches. However, real vectors inherently neglect the complex relation structures and lack the expressiveness of embeddings; hence, they may guide the embeddings to be falsely generated which results in alignment performance degradation. To overcome these problems, we propose a novel KG alignment framework, ComplexGCN, which learns the embeddings of both entities and relations in complex spaces while capturing both semantic and neighborhood information simultaneously. The proposed model ensures richer expressiveness and more accurate embeddings by successfully capturing various relation structures in complex spaces with high‐level computation. The model further incorporates relation label and direction information with a low degree of freedom. To compare our proposal against the state‐of‐the‐art baseline techniques, we conducted extensive experiments on real‐world datasets. The empirical results show the efficiency and effectiveness of the proposed method.

From diagnosis to therapies, the development of artificial intelligence (AI) has facilitated improvements in personalised healthcare applications. The evolution of AI in healthcare is closely related to the changes in the types and volumes of data which we need to deal with. The first generation of healthcare technologies, represented by the highly successful relational databases, are designed to handle structured data involving patient demographics, patient care, treatments, and outcomes of those treatments. Big Data platforms, which are representative of the current mainstream healthcare technologies, are built to process unstructured data from sources like electronic health records, medical imaging, genomic sequencing, and pharmaceutical research. The next generation of healthcare technologies will potentially be Edge-of-Things data, represented by massive amount of streaming data generated from Internet-of-Things frameworks, Cloud systems, and Edge computing platforms.

Network alignment is the task of identifying topologically and semantically similar nodes across (two) different networks. It plays an important role in various applications ranging from social network analysis to bioinformatic network interactions. However, existing alignment models either cannot handle large-scale graphs or fail to leverage different types of network information or modalities. In this paper, we propose a novel end-to-end alignment framework that can leverage different modalities to compare and align network nodes in an efficient way. In order to exploit the richness of the network context, our model constructs multiple embeddings for each node, each of which captures one modality or type of network information. We then design a late-fusion mechanism to combine the learned embeddings based on the importance of the underlying information. Our fusion mechanism allows our model to be adapted to various types of structure of the input network. Experimental results show that our technique outperforms state-of-the-art approaches in terms of accuracy on real and synthetic datasets, while being robust against various noise factors.

Location-based social networks (LBSNs) have emerged over the past few years. Their exponential network effects depend on the fact that each user can share her daily digital footprints with different communities, in different places, and at different times (for example in the form of check-in activities). Unlike other types of social networks, activities in an LBSN can potentially be performed by several users in a collaborative way. Existing studies of representation learning for LBSNs often consider them as regular graphs and ignore these high-order, dynamic, and multi-role contexts, since their holistic interactions are quite difficult to capture. In this paper, we propose a model in which these holistic interactions can be learned and transferred into node embeddings derived from a hypergraph representation and a persona decomposition process. More specifically, the model learns from friendship edges, check-in hyperedges, and node personas at the same time, and devises multiple presentations for each user that reflects their multiple roles in a social context. The embedding learning process also exploits useful patterns such as user co-location and sequential effects through a carefully designed point-of-interest splitting step. Extensive experiments on real and synthetic datasets show that our model outperforms alternative state-of-the-art embedding methods on friendship and location prediction tasks.

With recent advancements in graph neural networks (GNN), GNN-based recommender systems (gRS) have achieved remarkable success in the past few years. Despite this success, existing research reveals that gRSs are still vulnerable to poison attacks , in which the attackers inject fake data to manipulate recommendation results as they desire. This might be due to the fact that existing poison attacks (and countermeasures) are either model-agnostic or specifically designed for traditional recommender algorithms (e.g., neighborhood-based, matrix-factorization-based, or deep-learning-based RSs) that are not gRS. As gRSs are widely adopted in the industry, the problem of how to design poison attacks for gRSs has become a need for robust user experience. Herein, we focus on the use of poison attacks to manipulate item promotion in gRSs. Compared to standard GNNs, attacking gRSs is more challenging due to the heterogeneity of network structure and the entanglement between users and items. To overcome such challenges, we propose GSPAttack —a generative surrogate-based poison attack framework for gRSs. GSPAttack tailors a learning process to surrogate a recommendation model as well as generate fake users and user-item interactions while preserving the data correlation between users and items for recommendation accuracy. Although maintaining high accuracy for other items rather than the target item seems counterintuitive, it is equally crucial to the success of a poison attack. Extensive evaluations on four real-world datasets revealed that GSPAttack outperforms all baselines with competent recommendation performance and is resistant to various countermeasures.

Remote screening of respiratory diseases has been widely studied as a non-invasive and early instrument for diagnosis purposes, especially in the pandemic. The respiratory sound classification task has been realized with numerous deep neural network (DNN) models due to their superior performance. However, in the high-stake medical domain where decisions can have significant consequences, it is desirable to develop interpretable models; thus, providing understandable reasons for physicians and patients. To address the issue, we propose a prototype learning framework, that jointly generates exemplar samples for explanation and integrates these samples into a layer of DNNs. The experimental results indicate that our method outperforms the state-of-the-art approaches on the largest public respiratory sound database.

Network alignment is the task of identifying topo-logically and semantically similar nodes across (two) different networks. However, existing alignment models either cannot handle large-scale graphs or fail to leverage different types of network information or modalities. In this paper, we pro-pose a novel end-to-end alignment framework that can lever-age different modalities to compare and align network nodes in an efficient way. A comprehensive evaluation on various datasets shows that our technique outperforms state-of-the-art approaches. Our source code is available at https://github.com/thanhtrunghuynh93/holisticEmbeddingsNA.

Hundreds of thousands of rumours emerge every day. Algorithmic models shall therefore support users of social platforms and provide alerts to prevent users from accidentally spreading rumours. However, existing alerting mechanisms are limited to post-hoc classification, and rumours are often detected after the damage has been done. This paper presents exRumourLens, a system that enables tracking and auditing of potential rumours as they emerge. To this end, it identifies local anomalies related to individual entities, as well as global anomalies on the level of subgraphs of a network of entities. exRumourLens provides various views on such local and global anomalies, thereby providing detailed explanations on emerging rumours and supporting their critical exploration. The source code is available at https://rumourlens.github.io/.

Respiratory sound classification is an important tool for remote screening of respiratory-related diseases such as pneumonia, asthma, and COVID-19. To facilitate the interpretability of classification results, especially ones based on deep learning, many explanation methods have been proposed using prototypes. However, existing explanation techniques often assume that the data is non-biased and the prediction results can be explained by a set of prototypical examples. In this work, we develop a unified example-based explanation method for selecting both representative data (prototypes) and outliers (criticisms). In particular, we propose a novel application of adversarial attacks to generate an explanation spectrum of data instances via an iterative fast gradient sign method. Such unified explanation can avoid over-generalisation and bias by allowing human experts to assess the model mistakes case by case. We performed a wide range of quantitative and qualitative evaluations to show that our approach generates effective and understandable explanation and is robust with many deep learning models.

The heterogeneity of today’s Web sources requires information retrieval (IR) systems to handle multi-modal queries. Such queries define a user’s information needs by different data modalities, such as keywords, hashtags, user profiles, and other media. Recent IR systems answer such a multi-modal query by considering it as a set of separate uni-modal queries. However, depending on the chosen operationalisation, such an approach is inefficient or ineffective. It either requires multiple passes over the data or leads to inaccuracies since the relations between data modalities are neglected in the relevance assessment. To mitigate these challenges, we present an IR system that has been designed to answer genuine multi-modal queries. It relies on a heterogeneous network embedding, so that features from diverse modalities can be incorporated when representing both, a query and the data over which it shall be evaluated. By embedding a query and the data in the same vector space, the relations across modalities are made explicit and exploited for more accurate query evaluation. At the same time, multi-modal queries are answered with a single pass over the data. An experimental evaluation using diverse real-world and synthetic datasets illustrates that our approach returns twice the amount of relevant information compared to baseline techniques, while scaling to large multi-modal databases.

Knowledge graphs (KGs) have become popular structures for unifying real-world entities by modelling the relationships between them and their attributes. To support multilingual applications, a significant number of language-specific KGs have been built by different parties using various data sources. As a result, these monolingual KGs are often disconnected, causing semantic heterogeneity and detracting from the original purpose of KGs. Entity alignment – the task of identifying corresponding entities across different KGs – has attracted a great deal of attention in both academia and industry. However, existing alignment techniques often require large amounts of labelled data, are unable to encode multi-modal data simultaneously, and enforce only a few consistency constraints. In this paper, we propose an end-to-end, unsupervised entity alignment framework for cross-lingual KGs that fuses different types of information in order to fully exploit the richness of KG data. The model captures the relation-based correlation between entities by using a multi-order graph convolutional neural (GCN) model that is designed to satisfy the consistency constraints, while incorporating the attribute-based correlation via a translation machine. We adopt a late-fusion mechanism to combine all the information together, which allows these approaches to complement each other and thus enhances the final alignment result, and makes the model more robust to consistency violations. Empirical results for various scenarios on real-world and synthetic KGs show that our model is up to 22.71 percent more accurate and orders of magnitude faster than existing baselines. We also demonstrate its sensitivity to hyper-parameters, effort saving in terms of labelling, and the robustness against adversarial conditions.

Knowledge graph (KG) entity alignment is the task of identifying corresponding entities across different KGs. Existing alignment techniques often require large amounts of labelled data, are unable to encode multi-modal data simultaneously, and enforce only a few consistency constraints. In this paper, we propose an end-to-end, unsupervised entity alignment framework for cross-lingual KGs using multi-order graph convolutional networks. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of accuracy, efficiency, and label saving.

To support reactive and predictive applications, complex event processing (CEP) systems detect patterns in event streams based on predefined queries. To determine the events that constitute a query match, their payload data may need to be assessed together with data from remote sources. Such dependencies are problematic, since waiting for remote data to be fetched interrupts the processing of the stream. Yet, without event selection based on remote data, the query state to maintain may grow exponentially. In either case, the performance of the CEP system degrades drastically.

Network alignment is the problem of pairing nodes between two graphs such that the paired nodes are structurally and semantically similar. A well-known application of network alignment is to identify which accounts in different social networks belong to the same person. Existing alignment techniques, however, lack scalability, cannot incorporate multi-dimensional information without training data, and are limited in the consistency constraints enforced by an alignment. In this paper, we propose a fully unsupervised network alignment framework based on a multi-order embedding model. The model learns the embeddings of each node using a graph convolutional neural representation, which we prove to satisfy consistency constraints. We further design a data augmentation method and a refinement mechanism to make the model adaptive to consistency violations and noise. Extensive experiments on real and synthetic datasets show that our model outperforms state-of-the-art alignment techniques. We also demonstrate the robustness of our model against adversarial conditions, such as structural noises, attribute noises, graph size imbalance, and hyper-parameter sensitivity.

The open nature of the Web enables users to produce and propagate any content without authentication, which has been exploited to spread thousands of unverified claims via millions of online documents. Maintenance of credible knowledge bases thus has to rely on fact checking that constructs a trusted set of facts through credibility assessment. Due to an inherent lack of ground truth information and language ambiguity, fact checking cannot be done in a purely automated manner without compromising accuracy. However, state-of-the-art fact checking services, rely mostly on human validation, which is costly, slow, and non-transparent. This paper presents FactCatch, a human-in-the-loop system to guide users in fact checking that aims at minimisation of the invested effort. It supports incremental quality estimation, mistake mitigation, and pay-as-you-go instantiation of a high-quality fact database.

Data models capture the structure and characteristic properties of data entities, e.g., in terms of a database schema or an ontology. They are the backbone of diverse applications, reaching from information integration , through peer-to-peer systems and electronic commerce to social networking . Many of these applications involve models of diverse data sources. Effective utilisation and evolution of data models, therefore, calls for matching techniques that generate correspondences between their elements. Various such matching tools have been developed in the past. Yet, their results are often incomplete or erroneous, and thus need to be reconciled, i.e., validated by an expert. This paper analyses the reconciliation process in the presence of large collections of data models, where the network induced by generated correspondences shall meet consistency expectations in terms of integrity constraints. We specifically focus on how to handle data models that show some internal structure and potentially differ in terms of their assumed level of abstraction. We argue that such a setting calls for a probabilistic model of integrity constraints, for which satisfaction is preferred, but not required. In this work, we present a model for probabilistic constraints that enables reasoning on the correctness of individual correspondences within a network of data models, in order to guide an expert in the validation process. To support pay-as-you-go reconciliation, we also show how to construct a set of high-quality correspondences, even if an expert validates only a subset of all generated correspondences. We demonstrate the efficiency of our techniques for real-world datasets comprising database schemas and ontologies from various application domains.

Social platforms became a major source of rumours. While rumours can have severe real-world implications, their detection is notoriously hard: Content on social platforms is short and lacks semantics; it spreads quickly through a dynamically evolving network; and without considering the context of content, it may be impossible to arrive at a truthful interpretation. Traditional approaches to rumour detection, however, exploit solely a single content modality, e.g., social media posts, which limits their detection accuracy. In this paper, we cope with the aforementioned challenges by means of a multi-modal approach to rumour detection that identifies anomalies in both, the entities (e.g., users, posts, and hashtags) of a social platform and their relations. Based on local anomalies, we show how to detect rumours at the network level, following a graph-based scan approach. In addition, we propose incremental methods, which enable us to detect rumours using streaming data of social platforms. We illustrate the effectiveness and efficiency of our approach with a real-world dataset of 4M tweets with more than 1000 rumours.

The Web constitutes a valuable source of information. In recent years, it fostered the construction of large-scale knowledge bases, such as Freebase, YAGO, and DBpedia. The open nature of the Web, with content potentially being generated by everyone, however, leads to inaccuracies and misinformation. Construction and maintenance of a knowledge base thus has to rely on fact checking, an assessment of the credibility of facts. Due to an inherent lack of ground truth information, such fact checking cannot be done in a purely automated manner, but requires human involvement. In this paper, we propose a comprehensive framework to guide users in the validation of facts, striving for a minimisation of the invested effort. Our framework is grounded in a novel probabilistic model that combines user input with automated credibility inference. Based thereon, we show how to guide users in fact checking by identifying the facts for which validation is most beneficial. Moreover, our framework includes techniques to reduce the manual effort invested in fact checking by determining when to stop the validation and by supporting efficient batching strategies. We further show how to handle fact checking in a streaming setting. Our experiments with three real-world datasets demonstrate the efficiency and effectiveness of our framework: A knowledge base of high quality, with a precision of above 90%, is constructed with only a half of the validation effort required by baseline techniques.

Participatory sensing has become a new data collection paradigm that leverages the wisdom of the crowd for big data applications without spending cost to buy dedicated sensors. It collects data from human sensors by using their own devices such as cell phone accelerometers, cameras, and GPS devices. This benefit comes with a drawback: human sensors are arbitrary and inherently uncertain due to the lack of quality guarantee. Moreover, participatory sensing data are time series that exhibit not only highly irregular dependencies on time but also high variance between sensors. To overcome these limitations, we formulate the problem of validating uncertain time series collected by participatory sensors. In this article, we approach the problem by an iterative validation process on top of a probabilistic time series model. First, we generate a series of probability distributions from raw data by tailoring a state-of-the-art dynamical model, namely Generalised Auto Regressive Conditional Heteroskedasticity (GARCH), for our joint time series setting. Second, we design a feedback process that consists of an adaptive aggregation model to unify the joint probabilistic time series and an efficient user guidance model to validate aggregated data with minimal effort. Through extensive experimentation, we demonstrate the efficiency and effectiveness of our approach on both real data and synthetic data. Highlights from our experiences include the fast running time of a probabilistic model, the robustness of an aggregation model to outliers, and the significant effort saving of a guidance model.

What-if analysis is a data-intensive exploration to inspect how changes in a set of input parameters of a model influence some outcomes. It is motivated by a user trying to understand the sensitivity of a model to a certain parameter in order to reach a set of goals that are defined over the outcomes. To avoid an exploration of all possible combinations of parameter values, efficient what-if analysis calls for a partitioning of parameter values into data ranges and a unified representation of the obtained outcomes per range. Traditional techniques to capture data ranges, such as histograms, are limited to one outcome dimension. Yet, in practice, what-if analysis often involves conflicting goals that are defined over different dimensions of the outcome. Working on each of those goals independently cannot capture the inherent trade-off between them. In this paper, we propose techniques to recommend data ranges for what-if analysis, which capture not only data regularities, but also the trade-off between conflicting goals. Specifically, we formulate a parametric data partitioning problem and propose a method to find an optimal solution for it. Targeting scalability to large datasets, we further provide a heuristic solution to this problem. By theoretical and empirical analyses, we establish performance guarantees in terms of runtime and result quality.

Crowdsourcing has been widely established as a means to enable human computation at large-scale, in particular for tasks that require manual labelling of large sets of data items. Answers obtained from heterogeneous crowd workers are aggregated to obtain a robust result. However, existing methods for answer aggregation are designed for \\emph{discrete} tasks, where answers are given as a single label per item. In this paper, we consider \\emph{partial-agreement} tasks that are common in many applications such as image tagging and document annotation, where items are assigned sets of labels. Going beyond the state-of-the-art, we propose a novel Bayesian nonparametric model to aggregate the partial-agreement answers in a generic way. This model enables us to compute the consensus of partially-sound and partially-complete worker answers, while taking into account mutual relations in labels and different answer sets. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of precision, recall, and scalability.

Crowdsourcing has been widely established as a means to enable human computation at large-scale, in particular for tasks that require manual labelling of large sets of data items. Answers obtained from heterogeneous crowd workers are aggregated to obtain a robust result. However, existing methods for answer aggregation are designed for discrete tasks, where answers are given as a single label per item. In this paper, we consider-partial-agreement-tasks that are common in many applications such as image tagging and document annotation, where items are assigned sets of labels. Common approaches for the aggregation of partial-agreement answers either (i) reduce the problem to several instances of an aggregation problem for discrete tasks or (ii) consider each label independently. Going beyond the state-of-the-art, we propose a novel Bayesian nonparametric model to aggregate the partial-agreement answers in a generic way. This model enables us to compute the consensus of partially-sound and partially-complete worker answers, while taking into account mutual relationships in labels and different answer sets. We also show how this model is instantiated for incremental learning, incorporating new answers from crowd workers as they arrive. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of precision, recall, and robustness against faulty workers and data sparsity.

Today's social platforms, such as Twitter and Facebook, continuously generate massive volumes of data. The resulting data streams exceed any reasonable limit for permanent storage, especially since data is often redundant, overlapping, sparse, and generally of low value. This calls for means to retain solely a small fraction of the data in an online manner. In this paper, we propose techniques to effectively decide which data to retain, such that the induced loss of information, the regret of neglecting certain data, is minimized. These techniques enable not only efficient processing of massive streaming data, but are also adaptive and address the dynamic nature of social media. Experiments on large-scale real-world datasets illustrate the feasibility of our approach in terms of both, runtime and information quality.

Information visualization is essential for improving effectiveness and efficiency of data exploration and knowledge discovery. Therefore, visualization has been used in a wide range of fields from biology, medicine, criminal activity analysis to business and education. Information visualization has become more important than ever as the amount of data being generated has increased dramatically in recent years. One of the major difficulties of information visualization is performance, and this is even more critical when visualizing big data. One potential solution to this challenge is data sampling while maintaining fidelity of visual representation. In this paper, we propose two new centrality clustering-based sampling approaches that apply centrality measures on clusters of data points in order to make more informed sampling than random sampling approaches. We evaluate the new methods on graph data sets. The results show that the new methods significantly outperform existing data sampling methods in term of perceived differences and their ability to preserve essential visual information. Moreover computational complexity is comparable or even better than simple random sampling methods.

Learning now occurs in various manners in social networks, utilizing practice communities and learning networks. In this context, students are interested in exploring learning activities of other students without having to read through large quantities of textual content. Students tend to be interested in finding information concerning their majors, contents of their subjects and their co-learners. However, with the huge amount of information available, it is difficult for users to identify relevant information. Visualization comes in to help improve the situation. In this paper, we identify opportunities for deploying visualization in educational social networks to enhance learners' efficiency, effectiveness and learning experiences. We present a new learning space model designed to capture learning activities in educational social networks. A new 3D visualization system for the learning space model is also developed to help users obtain a better understanding of learning activities in social networks. We evaluate our approaches on educational social networks using real data. The evaluation results show that our approach of visualizing learning activities helps learners grasp a better overview of their study environment. Also, the learning process is perceived as easier, more engaging and more fun for the users, compared to traditional ways of displaying information.

Schema matching supports data integration by establishing correspondences between the attributes of independently designed database schemas. In recent years, various tools for automatic pair-wise matching of schemas have been developed. Since the matching process is inherently uncertain, the correspondences generated by such tools are often validated by a human expert. In this work, we consider scenarios in which attribute correspondences are identified in a network of schemas and not only in a pairwise setting. Here, correspondences between different schemas are interrelated, so that incomplete and erroneous matching results propagate in the network and the validation of a correspondence by an expert has ripple effects. To analyse and reconcile such matchings in schema networks, we present the Schema Matching Analyzer and Reconciliation Tool (SMART). It allows for the definition of network-level integrity constraints for the matching and, based thereon, detects and visualizes inconsistencies of the matching. The tool also supports the reconciliation of a matching by guiding an expert in the validation process and by offering semi-automatic conflict-resolution techniques.

The amount of information available on the Web has been growing dramatically, raising the importance of techniques for searching the Web. Recently, Web Tables emerged as a model, which enables users to search for information in a structured way. However, effective presentation of results for Web Table search requires (1) selecting a ranking of tables that acknowledges the diversity within the search result; and (2) summarizing the information content of the selected tables concisely but meaningful. In this paper, we formalize these requirements as the diversified table selection problem and the structured table summarization problem. We show that both problems are computationally intractable and, thus, present heuristic algorithms to solve them. For these algorithms, we prove salient performance guarantees, such as near-optimality, stability, and fairness. Our experiments with real-world collections of thousands of Web Tables highlight the scalability of our techniques. We achieve improvements up to 50% in diversity and 10% in relevance over baselines for Web Table selection, and reduce the information loss induced by table summarization by up to 50%. In a user study, we observed that our techniques are preferred over alternative solutions.

Schema matching is the process of establishing correspondences between the attributes of database schemas for data integration purposes. Although several automatic schema matching tools have been developed, their results are often incomplete or erroneous. To obtain a correct set of correspondences, a human expert is usually required to validate the generated correspondences. We analyze this reconciliation process in a setting where a number of schemas needs to be matched, in the presence of consistency expectations about the network of attribute correspondences. We develop a probabilistic model that helps to identify the most uncertain correspondences, thus allowing us to guide the expert's work and collect his input about the most problematic cases. As the availability of such experts is often limited, we develop techniques that can construct a set of good quality correspondences with a high probability, even if the expert does not validate all the necessary correspondences. We demonstrate the efficiency of our techniques through extensive experimentation using real-world datasets.

As the volumes of AI problems involving human knowledge are likely to soar, crowdsourcing has become essential in a wide range of world-wide-web applications. One of the biggest challenges of crowdsourcing is aggregating the answers collected from crowd workers; and thus, many aggregate techniques have been proposed. However, given a new application, it is difficult for users to choose the best-suited technique as well as appropriate parameter values since each of these techniques has distinct performance characteristics depending on various factors (e.g. worker expertise, question difficulty). In this paper, we develop a benchmarking tool that allows to (i) simulate the crowd and (ii) evaluate aggregate techniques in different aspects (accuracy, sensitivity to spammers, etc.). We believe that this tool will be able to serve as a practical guideline for both researchers and software developers. While researchers can use our tool to assess existing or new techniques, developers can reuse its components to reduce the development complexity.