The manufacturing business field is accommodating a massive number of networked sensors, which are offering a new horizon of microscopic observability; every single piece of the production line is enabled to be monitored, transformed and organized into collective digital assets. Intensive analytics on the digital assets potentially offers new business solutions to improve the manufacturing productivity and secure the business continuity. For example, the capability of exhaustive production inspection would potentially remove the delivery of defective products. Another scenario is the capability of expeditiously identifying an impaired part in the production line, which would significantly reduce the yield loss. These business analytics work often hold the nature that they need to precisely identify a particular manufacturing event of interest (e.g., producing an intermediate material at a suspiciously faulty machine) and then trace back or forward every ancestor or descendant of the event thoroughly in the manufacturing pipeline spanning from the initial raw material to the final product. Yet, such microscopic business traceability has not been actively addressed in the technology community. This paper presents our experimental exploration of how to design relational database for manufacturing business analytics, which is technically differentiated by the involvement of microscopic business traceability from traditional business analytics. We have performed intensive experiments and cost-performance analyses to figure out the trade-offs associated with major options of physical database design with different benchmarks on a commercial database management system (DBMS) in an enterprise environment. The combination of columnar table stores and secondary indexes is recommended for manufacturing business analytics workloads, even though the pure use of columnar table stores is recommended for traditional analytics workloads. We believe that our experience would help those planning to work on microscopic business tractability in manufacturing and other fields.

Association-rule mining has heretofore relied on the condition of high support to do its work efficiently. In particular, the well-known a priori algorithm is only effective when the only rules of interest are relationships that occur very frequently. However, there are a number of applications, such as data mining, identification of similar Web documents, clustering, and collaborative filtering, where the rules of interest have comparatively few instances in the data. In these cases, we must look for highly correlated items, or possibly even causal relationships between infrequent items. We develop a family of algorithms for solving this problem, employing a combination of random sampling and hashing techniques. We provide analysis of the algorithms developed and conduct experiments on real and synthetic data to obtain a comparative performance analysis.

Association rule mining has heretofore relied on the condition of high support to do its work efficiently. In particular, the well-known a-priori algorithm is only effective when the only rules of interest are relationships that occur very frequently. However, there are a number of applications, such as data mining, identification of similar Web documents, clustering and collaborative filtering, where the rules of interest have comparatively few instances in the data. In these cases, we must look for highly correlated items, or possibly even causal relationships between infrequent items. We develop a family of algorithms for solving this problem, employing a combination of random sampling and hashing techniques. We provide an analysis of the algorithms developed and conduct experiments on real and synthetic data to obtain a comparative performance analysis.

Dynamic miss-counting (DMC) algorithms are proposed which find all implication and similarity rules with confidence pruning but without support pruning. To handle data sets with a large number of columns, we propose dynamic pruning techniques that can be applied during data scanning. DMC counts the numbers of rows in which each pair of columns disagree instead of counting the number of hits. DMC deletes a candidate as soon as the number of misses exceeds the maximum number of misses allowed for that pair. We also propose several optimization techniques that reduce the required memory size significantly. We evaluated our algorithms by using four data sets, viz. Web access logs, a Web page-link graph, news documents and a dictionary. These data sets have between 74,000 and 700,000 items. Experiments show that DMC can find high-confidence rules for such large data sets efficiently.