Frequent pattern mining has gained popularity in the realm of knowledge discovery and big data analytics as it identifies sets of items that frequently co-occur (e.g., popular merchandise items or social events). In general, frequent pattern mining can be broadly classified into two categories: (i) transaction-centric algorithms that mines frequent patterns horizontally and (ii) item-centric mining algorithms that mines frequent patterns vertically. Irrespective of their categories, traditional frequent pattern mining algorithms aim to find Boolean frequent patterns, revealing whether some specific items are present in (or absent from) the discovered patterns. In the context of social network mining and analytics, Boolean frequent pattern algorithms can help reveal whether a social entity follows another in a network or on a social networking site. However, in numerous real-life applications, quantities of items within patterns become essential. For example, the quantity of followed items (e.g., like posts) can significantly influence the social interactions between entities in a network. In this paper, we present a social network mining and analytics algorithm---called QSN---for discovering quantitative frequent patterns from social networks. The algorithm represents the big data as a collection of item-centric bitmaps, each capturing the absence or presence of a transaction containing the item, along with the quantity of that item in each transaction. Subsequently, it vertically mines quantitative frequent patterns, strategically avoiding the generation of an excessive number of redundant candidate patterns, thereby accelerating the mining process. Results of our evaluation demonstrate the superiority of our QSN algorithm over the existing horizontal quantitative frequent pattern algorithm called MQA-M, highlighting its efficacy in social network mining and analytics.

Frequent pattern mining has been extended to the mining of other useful patterns. These include high-utility patterns. Many traditional high-utility mining algorithms focus on algorithmic efficiency when mining high-utility patterns from static databases. These algorithms rely on an assumption that the unit utility for a given item is a constant. However, as we are living in dynamic world where the unit utility (external unit profit) may change over time, such an assumption may not truly reflect reality in the real world. However, to the best of our knowledge, not a lot of works were done on mining dynamic profit from data streams yet. The emergence of big data has led to some performance challenges such that proper big data management techniques are needed for knowledge discovery from dynamic data streams. Traditional static data mining algorithms cannot directly apply to dynamic data. Furthermore, information in the data stream might not be uniformly distributed so it introduces extra challenges to process the data. Using big data stream processing platforms is necessary when mining real-world data stream. Leveraging the big data processing framework requires having scalable algorithms. In this paper, we present an enhanced high-utility data stream algorithm—called EHUI-Stream—to speed up the execution time and reduce memory usage. Utilizing our proposed algorithm, the data stream mining performance is expected to be further enhanced against both real-world datasets and synthetic datasets. Evaluation results on real-life data demonstrate the effectiveness of our platform in scalable high-utility pattern mining for dynamic profit from data streams for social and behavioral analytics.

The use of mobile phones, particularly smartphones, has been growing exponentially in recent times. From 2016 to 2021, smartphone users increased by more than 70%. With the increase in the popularity of smartphones, smartphones have become the prime target for criminal hackers. As a result, Android malware samples are coming to the market at an alarming rate. A study shows that there are more than 4 million malicious Android apps in the market, and each day around 11,000 new malwares add to this number. To combat this mass number of malware, we need a malware detection system that is efficient in detecting malicious Android apps. There are numerous existing malware detection systems, but most of them require countless features from both dynamic and static analysis. Thus, they are not scalable, lightweight, and efficient in detecting malware. Additionally, most studies that used limited features like only permission data, had done their research on much older dataset. Hence, there is a need for new research on this topic. In this paper, we build a permission-based malware detector for Android application with a new dataset and significantly less permissions. Initially, we used support vector machine (SVM) and all the extracted permission data as features to build our classification model. The model accuracy, precision, recall and F1 score were 97.41 percent, which is higher than the other state of the art similar approaches done on an older dataset. Next, we replicated this similar study with a few different machine learning algorithms: random forest, decision tree and logistic regression, and observed they all give similar results. However, tree-based algorithm performs a little better than the other algorithms. Finally, to achieve a lightweight malware detection system, we reduced the number of permissions or the features on a two-step process, and found only a slight difference in results. In the first step, even after reducing the number of permissions by about 94%, the accuracy dropped by only 2.7%. In the second step, we further reduced the number of features or permissions and observed the difference in results. We managed to prune to 9 permissions while maintaining accuracy of 93%, which is lower than technique mentioned in other literature to reduce features.

Big data are everywhere. World Wide Web is an example of these big data. It has become a vast data production and consumption platform, at which threads of data evolve from multiple devices, by different human interactions, over worldwide locations, under divergent distributed settings. Embedded in these big web data is implicit, previously unknown and potentially useful information and knowledge that awaited to be discovered. This calls for web intelligence solutions, which make good use of data science and data mining (especially, web mining or social network mining) to discover useful knowledge and important information from the web. As a web mining task, web structure mining aims to examine incoming and outgoing links on web pages and make recommendations of frequently referenced web pages to web surfers. As another web mining task, web usage mining aims to examine web surfer patterns and make recommendations of frequently visited pages to web surfers. While the size of the web is huge, the connection among all web pages may be sparse. In other words, the number of vertex nodes (i.e., web pages) on the web is huge, the number of directed edges (i.e., incoming and outgoing hyperlinks between web pages) may be small. This leads to a sparse web. In this paper, we present a solution for interpretable mining of frequent patterns from sparse web. In particular, we represent web structure and usage information by bitmaps to capture connections to web pages. Due to the sparsity of the web, we compress the bitmaps, and use them in mining influential patterns (e.g., popular web pages). For explainability of the mining process, we ensure the compressed bitmaps are interpretable. Evaluation on real-life web data demonstrates the effectiveness, interpretability and practicality of our solution for interpretable mining of influential patterns from sparse web.

Nowadays, social networking is popular. As such, numerous social networking sites (e.g., Facebook, YouTube, Instagram) are generating very large volumes of social data rapidly. Valuable knowledge and information is embedded into these big social data. As the social network can be very sparse, it is awaiting to be (a) compressed via social network data compression and (b) analyzed and mined via social network analysis and mining. We present in this paper a solution for compressing and mining social networks. It gives an interpretable compressed representation of sparse social network, and discovers interesting patterns from the social network. Results of our evaluation show the effectiveness of our solution in explaining the compression and mining of the sparse social network data.