Abstract Class imbalance, where datasets often lack sufficient samples for minority classes, is a persistent challenge in machine learning. Existing solutions often generate synthetic data to mitigate this issue, but they typically struggle with complex data distributions, primarily because they focus on oversampling the minority class while neglecting the relationships with the majority class. To overcome these limitations, we propose the Contrastive Tabular Variational Autoencoder (CTVAE), which integrates conditional Variational Autoencoders with contrastive learning techniques. CTVAE excels at generating high-quality synthetic samples that capture the intricate data distributions of both minority and majority classes. Additionally, it can be seamlessly integrated with variants of the Synthetic Minority Oversampling Technique (SMOTE) for enhanced effectiveness. Experimental results demonstrate that CTVAE substantially improves classification performance on imbalanced datasets, offering a more robust and holistic solution to the class imbalance problem.

The Integrated Data Infrastructure (IDI) in New Zealand is a critical asset that integrates citizen data from various public and private organizations for population-level analyses. However, access restrictions within the IDI environment present challenges for fully utilizing its potential. This study examines synthetic data as a potential solution, offering a comprehensive framework for generating customizable and easily implementable synthetic data. The evaluation of multiple data synthesis algorithms considers statistical similarity, machine learning utility, and privacy concerns. The findings reveal that distance-based algorithms, like SMOTE, strike a balance between accuracy and computational cost, making them suitable for IDI. The study also identifies the need for a clear release guide for micro-level synthetic data and proposes exploring a fully automatic data evaluation pipeline in future research. Additionally, the study highlights opportunities enabled by synthetic data, such as familiarization with administrative datasets, reproducibility of studies, pilot analyses, and enhanced cross-domain collaboration. Overall, the proposed framework and findings offer valuable insights and guidance for synthetic data projects within the IDI, advancing synthetic data privacy research and facilitating reproducibility, collaboration, and data sharing in the IDI ecosystem.

k-nearest neighbors (k-NN) is a well-known classification algorithm that is widely used in different domains. Despite its simplicity, effectiveness and robustness, k-NN is limited by the use of the Euclidean distance as the similarity metric, the arbitrarily selected neighborhood size k, the computational challenge of high-dimensional data, and the use of the simple majority voting rule in class determination. We sought to address the last issue and proposed the Centroid Displacement-based k-NN algorithm, where centroid displacement is used for class determination. This paper presents a simple yet efficient variant of our previous work, named Ensemble Centroid Displacement-based k-NN, which leverages the homogeneity of the nearest neighbors of test instances. Extensive experiments on various real and synthetic datasets were conducted to show the effectiveness and robustness of the proposed algorithm. Our experimental results demonstrate that the proposed algorithm is able to enhance the classification performance of the standard k-NN algorithm and its variants and also improve the computational efficiency. The performance of our algorithm was consistent and robust for both balanced and imbalanced datasets.