This work mines big data in Sentinel-1 satellite images to unveil geographical patterns in offshore wind energy. We leverage unsupervised machine learning to extract insights from a 44GB open access dataset for decision support in wind farm orientations to guide stakeholders. It has broader impacts of overcoming climate change by enhancing renewable energy.

This work demonstrates DISCERN (Detection Image System with Commonsense Efficient Ranking Network), a novel generalizable task-ranking approach to improve human-robot collaboration via "discern"-ing with commonsense knowledge (CSK) derived from huge data repositories, augmented with image models and other everyday premises. It is an explainable, efficient solution useful to dynamic multipurpose robots.

This work leverages CODA TB, a groundbreaking dataset for a novel comprehensive method of early TB detection from medical big data. Departing from the erstwhile, we find mere cough duration less effective in TB prediction. We discover key demographic and clinical factors (e.g. heart rate, presenting symptoms) to be crucial in distinguishing TB cases, motivating comprehensive cough data analysis with enhanced screening.

Structured knowledge is important for many AI applications. Commonsense knowledge, which is crucial for robust human-centric AI, is covered by a small number of structured knowledge projects. However, they lack knowledge about human traits and behaviors conditioned on socio-cultural contexts, which is crucial for situative AI. This paper presents Candle, an end-to-end methodology for extracting high-quality cultural commonsense knowledge (CCSK) at scale. Candle extracts CCSK assertions from a huge web corpus and organizes them into coherent clusters, for 3 domains of subjects (geography, religion, occupation) and several cultural facets (food, drinks, clothing, traditions, rituals, behaviors). Candle includes judicious techniques for classification-based filtering and scoring of interestingness. Experimental evaluations show the superiority of the Candle CCSK collection over prior works, and an extrinsic use case demonstrates the benefits of CCSK for the GPT-3 language model. Code and data can be accessed at https://candle.mpi-inf.mpg.de/.

We propose an approach CSK-Detector for object detection and image categorization, well-suited for big data, by transferring commonsense knowledge from a knowledge base, augmented with premises and quantifiers. It is implemented for domestic robotics, especially with the motivation that next-generation and multipurpose domestic robots should be able to seamlessly discern environments for specific tasks without prior annotation of excessive images. CSK-Detector is evaluated on real data, yielding better results than deep learning without commonsense, while also providing an explainable approach. It broadly impacts human-robot collaboration and smart living.

All biological species undergo change over time due to the evolutionary process. These changes can occur rapidly and unpredictably. Due to their high potential to spread quickly, it is critical to be able to monitor changes and detect viral variants. Phylogenetic trees serve as good methods to study evolutionary relationships. Complex big data in biomedicine is plentiful in regards to viral data. In this paper, we analyze phylogenetic trees with reference to viruses and conduct dynamic programming using the Smith-Waterman algorithm, followed by hierarchical clustering. This methodology constitutes an intelligent approach for data mining, paving the way for examining variations in SARS-Cov-2, which in turn can help to discover knowledge potentially useful in biomedicine.

Mental health is an ever-growing issue of concern, especially in light of the COVID pandemic. In this context, we study big data from social media over a 7-year time span to gauge evolving perceptions of mental health, and discuss our research findings, potentially useful for decision support in healthcare. We deploy topic modeling and sentiment analysis to estimate public perceptions of mental health issues, focusing on Twitter as the social media site. We claim that it is important to consider polarity as well as subjectivity in sentiment analysis to comprehend two different aspects of sentiment, i.e. orientation in the emotion, and extent of fact vs. opinion. We assert that ranking via topic modeling is beneficial to fathom the relative importance of issues over the years. We harness tools/techniques from natural language processing and data mining to discover knowledge from big data on social media, related to mental health. Some of our findings reveal that the sentiment around mental health has remained positive overall, but has decreased since the beginning of the COVID pandemic. Major events, such as elections and the pandemic, greatly impact the conversation surrounding mental health. Some topics have remained consistent throughout the years. In other topics, the tone of the public discussions has shifted. The outcomes of our study would be useful to a variety of professionals, ranging from data scientists to epidemiologists and psychologists. This work impacts big healthcare data in general.

Domain-specific big data poses issues in knowledge discovery and decision support given its many Vs such as volume, variety, and visualization. In this work, we propose solutions via Ishikawa diagrams, JESS (Java Expert System Shell) with rule discovery, as well as visual analytics. We deploy data science and domain models, targeting engineering and scientific domains. Our results yield high accuracy, efficiency and cost-effectiveness.

Experimental results are often plotted as 2-dimensional graphical plots (aka graphs) in scientific domains depicting dependent versus independent variables to aid visual analysis of processes. Repeatedly performing laboratory experiments consumes significant time and resources, motivating the need for computational estimation. The goals are to estimate the graph obtained in an experiment given its input conditions, and to estimate the conditions that would lead to a desired graph. Existing estimation approaches often do not meet accuracy and efficiency needs of targeted applications. We develop a computational estimation approach called AutoDomainMine that integrates clustering and classification over complex scientific data in a framework so as to automate classical learning methods of scientists. Knowledge discovered thereby from a database of existing experiments serves as the basis for estimation. Challenges include preserving domain semantics in clustering, finding matching strategies in classification, striking a good balance between elaboration and conciseness while displaying estimation results based on needs of targeted users, and deriving objective measures to capture subjective user interests. These and other challenges are addressed in this work. The AutoDomainMine approach is used to build a computational estimation system, rigorously evaluated with real data in Materials Science. Our evaluation confirms that AutoDomainMine provides desired accuracy and efficiency in computational estimation. It is extendable to other science and engineering domains as proved by adaptation of its sub-processes within fields such as Bioinformatics and Nanotechnology.

This paper proposes a pilot approach based on the comparative analysis of supervised Machine Learning models coupled with basic Natural Language Processing concepts for classifying Functional and Non-Functional Requirements from huge collections of data relevant to the Requirements Engineering (RE) phase within software development. The publicly available PROMISE Software Engineering Repository dataset is used in the execution of this approach. Non-Functional Requirements are further classified into subclasses based on attributes they address since they are not directly related to the core functions of the concerned software. This overall research initiative helps to make the RE phase more efficient and reduces human effort in software development. It leverages Big Data in Software Engineering.

We devise a method to study coastal dune vegetation based on drone data orthomosaic mapping and machine learning algorithms to distinguish objects with spatial and color accuracy from numerous high-quality drone multispectral images. It allows accurate surveying on the density of coastal dune vegetation and potentially individual species. We thus analyze big data to develop tools for coastal resiliency and sustainability.

Security patterns are proven solutions to recurring problems in software development. The growing importance of secure software development has introduced diverse research efforts on security patterns that mostly focused on classification schemes, evolution and evaluation of the patterns. Despite a huge mature history of research and popularity among researchers, security patterns have not fully penetrated software development practices. Besides, software security education has not been benefited by these patterns though a commonly stated motivation is the dissemination of expert knowledge and experience. This is because the patterns lack a simple embodiment to help students learn about vulnerable code, and to guide new developers on secure coding. In order to address this problem, we propose to conduct intelligent data mining in the context of software engineering to discover learner-friendly software security patterns. Our proposed model entails knowledge discovery from large scale published real-world vulnerability histories in software applications. We harness association rule mining for frequent pattern discovery to mine easily comprehensible and explainable learner-friendly rules, mainly of the type "flaw implies fix" and "attack type implies flaw", so as to enhance training in secure coding which in turn would augment secure software development. We propose to build a learner-friendly intelligent tutoring system (ITS) based on the newly discovered security patterns and rules explored. We present our proposed model based on association rule mining in secure software development with the goal of building this ITS. Our proposed model and prototype experiments are discussed in this paper along with challenges and ongoing work.

Commonsense knowledge is a foundational cornerstone of artificial intelligence applications. Whereas information extraction and knowledge base construction for instance-oriented assertions, such as Brad Pitt's birth date, or Angelina Jolie's movie awards, has received much attention, commonsense knowledge on general concepts (politicians, bicycles, printers) and activities (eating pizza, fixing printers) has only been tackled recently. In this tutorial we present state-of-the-art methodologies towards the compilation and consolidation of such commonsense knowledge (CSK). We cover text-extraction-based, multi-modal and Transformer-based techniques, with special focus on the issues of web search and ranking, as of relevance to the WSDM community.

This paper addresses commonsense knowledge (CSK) to enhance human-robot collaboration (HRC) in large scale smart manufacturing. As big data in collaborative robotics grows, CSK in useful to achieve task optimization as depicted in our simulation studies and laboratory experiments, extendable to real-world applications.

We propose a novel scalable Web portal called LSOMP (Large Scale Ordinance Mining Portal) to analyze ordinances and their tweets (of the order of thousands and millions). It entails commonsense knowledge (CSK) and natural language processing (NLP), disseminating ordinance-tweet mining results via interactive graphics and Question Answering (QA).

The novel coronavirus (Covid-19) has spread rapidly amongst countries all around the globe. Compared to the rise in cases, there are few Covid-19 testing kits available. Due to the lack of testing kits for the public, it is useful to implement an automated AI-based E-health decision support system as a potential alternative method for Covid-19 detection. As per medical examinations, the symptoms of Covid-19 could be somewhat analogous to those of pneumonia, though certainly not identical. Considering the enormous number of cases of Covid-19 and pneumonia, and the complexity of the related images stored, the data pertaining to this problem of automated detection constitutes big data. With rapid advancements in medical imaging, the development of intelligent predictive and diagnostic tools have also increased at a rapid rate. Data mining and machine learning techniques are widely accepted to aid medical diagnosis. In this paper, a huge data set of X-ray images from patients with common bacterial pneumonia, confirmed Covid-19 disease, and normal healthy cases are utilized for AI-based decision support in detecting the Coronavirus disease. The transfer learning approach, which enables us to learn from a smaller set of samples in a problem and transfer the discovered knowledge to a larger data set, is employed in this study. We consider transfer learning using three different models that are pre-trained on several images from the ImageNet source. The models deployed here are VGG16, VGG19, and ResNet101. The dataset is generated by gathering different classes of images. We present our approach and preliminary evaluation results in this paper. We also discuss applications and open issues.

In the ever-growing data-driven world today, data increases in many forms, e.g. e-commerce sites uploading new products, streaming services adding TV shows and movies, and music platforms uploading new songs. It would be highly infeasible for end users to quickly browse all this data. Hence recommender systems can benefit end users (individuals as well as companies) in efficiently finding suitable products. Rather than making end users search through a vast array of items, recommender systems can suggest suitable items to users based on popularity of the items and the respective users' buying behavior. Accordingly, in this paper we explore two techniques widespread in recommender systems, i.e. item-based collaborative filtering and association rule mining, over Amazon review data on cellphones and accessories, and build a baseline recommender system scalable to larger data. Association rule mining is explored using the Apriori algorithm to find patterns in the data from transaction history. Item-based collaborative filtering is deployed using a correlation matrix to find similar products. Both these techniques yield useful results as evident from our baseline experiments. This work constitutes an exploratory study with longtime products in e-commerce and sets the stage for mining online data on relatively new products pertinent to the Covid-19 pandemic. These include face masks, hand sanitizers, disinfectant sprays, antibacterial wipes etc. Since multiple vendors are designing such crucial products today, it is important to provide recommendations to potential buyers. An ultimate goal in our work is to build a recommender app for e-commerce based on interesting results from our findings. This work constitutes intelligent data mining scalable over big data in e-commerce. It makes broader impacts on smart cities, since this fits the smart living and smart economy characteristics.

As big data heads towards big knowledge, data management and machine learning techniques work together to address several interesting problems. In this paper, we address a problem in natural language processing that involves learning by mining from large text databases. More specifically, we deal with the problem of preposition prediction, especially for ESL (English as a second language) learners. Prepositions are function words that typically show a relationship between a noun or a pronoun and other elements of a sentence. They play a key role in determining the meaning of a sentence. Accurate prediction of correct prepositions in a sentence is a challenging job since preposition usage is one of the most subtle aspects of the English grammar, making it difficult for non-native speakers. This paper proposes an approach for preposition prediction called WordPrep based on which we build a tool. WordPrep relies on mining based on the words themselves rather than on their lexical or syntactic connotations. This addresses the challenges of prepositions appearing in idiomatic phrases or in different semantic contexts, due to which the actual words are better than their grammatical positions. Our proposed solution entails a direct data-driven approach to predict the missing preposition in a sentence by learning from matching tokens consisting of ngrams with words before and after the preposition. Using various searches and pattern-matching methods against a large number of database records from big text corpora, this approach predicts the missing preposition(s). We describe our pilot approach, tool implementation and experiments in this paper. This work is particularly helpful for pedagogical applications.

Over the past few years, we have worked on pioneering an approach that employs Commonsense Knowledge (CSK) to automate the identification of Implicit Requirements (IMRs) from text in large Software Requirements Specifications (SRS) documents. This paper builds on our IMR-identification approach by adding CNN-based deep learning to detect IMRs from complex SRS big data such as images and tables.

This paper proposes an approach called TOLCS (Tweet Ordinance Linkage by Commonsense and Semantics) which helps connect specific ordinances or local laws to pertinent tweets expressing public reactions on them. TOLCS incorporates pragmatic aspects by commonsense knowledge, and semantic aspects by domain knowledge along with text similarity methods. It uses a blocking mechanism to reduce sample space for efficiently processing big data on ordinances and tweets.

PIKM workshop offers to Ph.D. students the possibility to bring their work to an international and interdisciplinary research community, and create a network of young researchers to exchange and develop new and promising ideas. Similarly to the CIKM, PIKM workshop covers a wide range of topics in the areas of databases, information retrieval and knowledge management.

Knowledge discovery from big data demands effective representation of data. However, big data are often characterized by high dimensionality, which makes knowledge discovery more difficult. Many techniques for dimensionality reudction have been proposed, including well-known Fisher's Linear Discriminant Analysis (LDA). However, the Fisher criterion is incapable of dealing with heteroscedasticity in the data. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed that is capable of exploiting heteroscedastic information in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In this article, we show precisely what can be expected from the Chernoff criterion. In particular, we show that the Chernoff criterion exploits the Fisher and Fukunaga-Koontz transforms in computing its linear discriminants. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.

The PIKM 2012 workshop is the 5th of its kind after 4 successful PhD workshops at ACM CIKM. This PhD workshop invites papers that describe the Ph.D. dissertation proposals of doctoral students in any of the CIKM areas: databases, information retrieval, data mining and knowledge management. Interdisciplinary work across these tracks is particularly encouraged. This year PIKM has received around 25 submissions from over 12 countries across the globe, among which 10 have been accepted as full papers for oral presentation while 4 have been accepted as short ones for poster presentation. The selection has been conducted based on reviews submitted by an expert team comprising 21 PC members spanning 12 countries and 6 continents with a good balance of industry and academia.

This paper demonstrates a system called SITAC based on our proposed approach to automate the discovery of concepts (called SITACs) in text sources that are identical semantically but alter their names over time. This system is developed to perform time-aware translation of queries over text corpora by incorporating terminology evolution, thus providing more accurate responses to users, e.g., query processing on Mumbai should automatically take into account its former name Bombay. The SITAC system constitutes a novel collaborative framework of natural language processing, association rule mining and contextual similarity.

The World Wide Web no longer consists just of HTML pages. Our work sheds light on a number of trends on the Internet that go beyond simple Web pages. The hidden Web provides a wealth of data in semi-structured form, accessible through Web forms and Web services. These services, as well as numerous other applications on the Web, commonly use XML, the eXtensible Markup Language. XML has become the lingua franca of the Internet that allows customized markups to be defined for specific domains. On top of XML, the Semantic Web grows as a common structured data source. In this work, we first explain each of these developments in detail. Using real-world examples from scientific domains of great interest today, we then demonstrate how these new developments can assist the managing, harvesting, and organization of data on the Web. On the way, we also illustrate the current research avenues in these domains. We believe that this effort would help bridge multiple database tracks, thereby attracting researchers with a view to extend database technology.

Well known linear discriminant analysis (LDA) based on the Fisher criterion is incapable of dealing with heteroscedasticity in data. However, in many practical applications we often encounter heteroscedastic data, i.e., within-class scatter matrices can not be expected to be equal. A technique based on the Chernoff criterion for linear dimensionality reduction has been proposed recently. The technique extends well-known Fisher's LDA and is capable of exploiting information about heteroscedasticity in the data. While the Chernoff criterion has been shown to outperform the Fisher's, a clear understanding of its exact behavior is lacking. In addition, the criterion, as introduced, is rather complex, making it difficult to clearly state its relationship to other linear dimensionality reduction techniques. In this paper, we show precisely what can be expected from the Chernoff criterion and its relations to the Fisher criterion and Fukunaga-Koontz transform. Furthermore, we show that a recently proposed decomposition of the data space into four subspaces is incomplete. We provide arguments on how to best enrich the decomposition of the data space in order to account for heteroscedasticity in the data. Finally, we provide experimental results validating our theoretical analysis.

Time-stamped documents such as newswire articles, blog posts and other web-pages are often archived online. When these archives cover long spans of time, the terminology within them could undergo significant changes. Hence, when users pose queries pertaining to historical information, over such documents, the queries need to be translated, taking into account these temporal changes, to provide accurate responses to users. For example, a query on Sri Lanka should automatically retrieve documents with its former name Ceylon. We call such concepts SITACs, i.e., Semantically Identical Temporally Altering Concepts. In order to discover SITACs, we propose an approach based on a novel framework constituting an integration of natural language processing, association rule mining, and contextual similarity as a learning technique. The proposed approach has been experimented with real data and has been found to yield good results with respect to efficiency and accuracy.

The PIKM workshop focuses on papers consisting mainly of the Ph.D. dissertation proposals of doctoral students. A wide range of topics on any area in databases, information retrieval and knowledge management are presented at this workshop. The areas of interest are similar to those at the CIKM main conference in the three respective tracks. Interdisciplinary work across these tracks is encouraged.

This paper describes a graphical data mining system called AutoDomainMine. It is based on our proposed approach of integrating clustering and classification to mine scientific data stored in a database. The data consists of input conditions of scientific experiments and graphs plotted as their results. This system mines the stored data in order to submit exact or approximate ranked responses to user queries intended to optimize the scientific processes.

In scientific domains, knowledge is often discovered from experiments by grouping or clustering them based on the similarity of their output. The causes of similarity are analyzed based on the input conditions characterizing a given type of output, i.e., a given cluster. This analysis helps in applications such as decision support in industry. Cluster representatives form at-a-glance depictions for such applications. Randomly selecting a set of conditions in a cluster as its representative is not sufficient since distinct combinations of inputs could lead to the same cluster. In this paper, an approach called DesCond is proposed to design semantics-preserving cluster representatives for scientific input conditions. We define a notion of distance for conditions to capture semantics based on the types of their attributes and their relative importance. Using this distance, methods of building candidate cluster representatives with different levels of detail are proposed. Candidates are compared using the DesCond Encoding proposed in this paper that assesses their complexity and information loss, given user interests. The candidate with the lowest encoding for each cluster is returned as its designed representative. DesCond is evaluated with real data from Materials Science. Evaluation with domain expert interviews and formal user surveys shows that designed representatives consistently outperform randomly selected ones and different candidates suit different users.