Soundex has been used for over a century for approximately matching records based on their phonetic footprint. In this paper, we examine a series of techniques a practitioner might employ in order to increase the algorithm’s matching capabilities, when utilizing Soundex for privacy preserving record linkage and a protocol based on Apache Spark, suitable for big data processing. We provide a detailed empirical assessment measuring matching quality and time performance of the proposed alternatives, showing that we achieve both precision and recall over 95% for large datasets in a few seconds and without utilizing any privacy-preserving blocking technique.

Record linkage, the process of identifying records representing the same real world entity in the absence of common unique identifiers, is one of the most intriguing problems in data processing, hence drawing attention for several decades. One approach for addressing this problem is by means of supervised learning. However, when it comes to linking records of individuals, the quality of the results may be low due to hidden bias phenomena, not deliberately caused, but originating from specific properties of the names of people of specific ethnic origin. In this paper, we focus on SVM-based record linkage and considering the fact of hidden bias, we propose a methodology oriented towards Ethnicity group-based training for specific Ethnicity groups, an approach that manages to elevate matching performance, compared to the average case.

The SARS-CoV-2 pandemic we are experiencing the last few years has resulted in a shift of many of our everyday life activities from the physical to the digital world. Education is a typical example of such a case with both tutors and students experiencing difficulties and delays in communication. To assist towards this direction, also considering the already forming future of online education, we present MI-OPJ, a microservices-based online programming judge.

In this work, we present the architecture of MILMS, an envisioned Microservices-based Learning Management System designed for deployment in the big data era. It is event-driven and it is designed for scalability, employing state-of-the-art methodologies and tools, taking into account the needs emerged when shifting from traditional to online learning, as in the situation occurred in our times.

When dealing with sensitive and personal user data, the process of record linkage raises privacy issues. Thus, privacy preserving record linkage has emerged with the goal of identifying matching records across multiple data sources while preserving the privacy of the individuals they describe. The task is very resource demanding, considering the abundance of available data, which, in addition, are often dirty. Blocking techniques are deployed prior to matching to prune out unlikely to match candidate records so as to reduce processing time. However, when scaling to large datasets, such methods often result in quality loss. To this end, we propose Multi-Sampling Transitive Closure for Encrypted Fields (MS-TCEF), a novel privacy preserving blocking technique based on the use of reference sets. Our new method effectively prunes records based on redundant assignments to blocks, providing better fault-tolerance and maintaining result quality while scaling linearly with respect to the dataset size. We provide a theoretical analysis on the method's complexity and show how it outperforms state-of-the-art privacy preserving blocking techniques with respect to both recall and processing cost.