The Adam optimizer is a popular choice in contemporary deep learning due to its strong empirical performance. However we observe that in privacy sensitive scenarios, the traditional use of Differential Privacy (DP) with the Adam optimizer leads to sub-optimal performance on several tasks. We find that this performance degradation is due to a DP bias in Adam's second moment estimator, introduced by the addition of independent noise in the gradient computation to enforce DP guarantees. This DP bias leads to a different scaling for low variance parameter updates, that is inconsistent with the behavior of non-private Adam, and Adam's sign descent interpretation. We propose the DP-AdamBC optimization algorithm, which corrects for the bias in the second moment estimation and retrieves the expected behaviour of Adam. Empirically, DP-AdamBC significantly improves the optimization performance of DP-Adam by up to 3.5% in final accuracy in image, text, and graph node classification tasks.

We present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMIA eliminates the common dependency of privacy measurement tools on in-distribution non-member data. As a result, PANORAMIA does not modify the model, training data, or training process, and only requires access to a subset of the training data. We evaluate PANORAMIA on ML models for image and tabular data classification, as well as on large-scale language models.

Wafer Bin Maps (WBM) frequently show various spatial failure patterns that provide crucial information for engineers to identify the root cause of failures and their consequent low yield. To shorten the root-cause diagnosis process, it is important to classify different failure patterns with high accuracy, especially when there is mixed type of failure patterns on the same wafer. The main challenges of mixed type classification in WBMs include: 1) Lack of accurately annotated real-world training dataset, 2) Imbalanced/long-tail distributions among classes, 3) Synthesized training data usually cannot reflect the practical application conditions. In this paper, we propose a weakly supervised learning approach and use an ensemble method based on triplet CNN models to classify mixed-type failure patterns in WBMs. We train the models based on the public WM-811K dataset, which is collected from real products but with only single-label annotations. We demonstrate that such models could mitigate the imbalanced class distribution and being able to learn efficiently from a weakly labeled dataset and achieve superior performances on the classification of real wafer maps with long-tail distributed mixed type failures. We also discuss the practical considerations of implementing such models and the advantages of using triplet over binary CNN models.