Building a Machine Learning model requires the use of large amounts of data. Due to privacy and regulatory concerns, these data might be owned by multiple sites and are often not mutually shareable. Our work deals with private learning and inference for the Weighted Random Forest model when data records are vertically distributed among multiple sites. Previous privacy-preserving vertical tree-based frameworks either adapt Secure Multi-party Computation or share intermediate results and are hard to generalize or scale. In contrast, our proposal contains efficient collaborative calculation algorithms of the Gini Index and Entropy for computing the impurity of decision tree nodes while protecting all intermediate values and disclosing minimal information. We offer a learning protocol based on the Paillier Cryptosystem and Digital Envelope. Also, we provide an inference protocol found on the Look-up Table. Our experiments show that the proposed protocols do not cause predictive performance loss while still establishing and utilizing the model within a reasonable time. The results imply that practitioners can overcome the barrier of data sharing and produce random forest models for data-heavy domains with strict privacy requirements, such as Health Prediction, Fraud Detection, and Risk Evaluation.

Supervised machine learning does not hold without data. However, the needed data can be distributed in different locations and are non-shareable under privacy constraints. Methods to circumvent disclosure restrictions in collaborative machine learning are in strong demand. Thus, we propose HERB+ (Homomorphic Encryption for Random forest and gradient Boosting plus), a confidential learning framework for tree-based models under the scenario of vertically dispersed data. While previous related work focused on a specific algorithm, this work presents a wide variety of privacy-preserved and distributed tree-based algorithms (i.e., Decision Tree, Random Forest, and Gradient Boosting Decision Trees for both classification and regression tasks). HERB+ provides the most detailed and general discussions on using Fully Homomorphic Encryption for computing distributed tree-based algorithms during the training process. Our experiments show that although the learning protocols' efficiencies are not optimal, the predictive performance and privacy are preserved. The results imply that practitioners can overcome the barrier of data sharing and produce tree-based models for data-heavy domains with strict privacy requirements, such as Health Prediction, Fraud Detection, and Risk Evaluation.