Text data are often encoded as dense vectors, known as embeddings, which capture semantic, syntactic, contextual, and domain-specific information. These embeddings, widely adopted in various applications, inherently contain rich information that may be susceptible to leakage under certain attacks. The GEIA framework highlights vulnerabilities in sentence embeddings, demonstrating that they can reveal the original sentences they represent. In this study, we reproduce GEIA's findings across various neural sentence embedding models. Additionally, we contribute new analysis to examine whether these models leak sensitive information from their training datasets. We propose a simple yet effective method without any modification to the attacker's architecture proposed in GEIA. The key idea is to examine differences between log-likelihood for masked and original variants of data that sentence embedding models have been pre-trained on, calculated on the embedding space of the attacker. Our findings indicate that following our approach, an adversary party can recover meaningful sensitive information related to the pre-training knowledge of the popular models used for creating sentence embeddings, seriously undermining their security. Our code is available on: https://github.com/taslanidis/GEIA

The accurate decoding of ElectroEncephaloGraphy (EEG) signals would bring us closer to understanding brain functionality, opening new pathways to fixing brain impairments and devising new Brain-Computer Interface (BCI)-related applications. The impressive success of deep convolutional neural networks extracting information from raw data in computer vision and natural language processing has motivated their investigation into EEG signal decoding. Consequently, a number of deep convolutional neural network models with state-of-the-art performance have been proposed in the literature for EEG signal decoding. However, because all these works aimed to find the model architecture with the best decoding accuracy, the model's size was left unbounded in that exploration. Considering the model size in the design of deep convolutional neural networks for EEG decoding could make their implementation on low-power microcontrollers (that may be integrated into a wearable system) with limited memory feasible. Thus, in this paper, we search for the most accurate deep convolutional neural network on the BCI Competition IV 2a dataset that can also fit in a microcontroller with less than 256KB SRAM. Specifically, we use a Neural Architecture Search (NAS) algorithm that considers, apart from the model's accuracy, the model size, the latency, and the peak memory utilization when running the model's inference. We compare our models with the model with the best decoding accuracy in the literature on the BCI Competition IV 2a dataset (baseline). We show that the discovered models could achieve similar accuracy to the baseline model while shrinking the memory footprint during inference by a factor of ≈ ×20, with a speedup in latency of up to ×1.7 on average.

A better understanding of ElectroEncephaloGraphy (EEG) and ElectroCorticoGram (ECoG) signals would get us closer to comprehending brain functionality, creating new avenues for treating brain abnormalities and developing novel Brain-Computer Interface (BCI)-related applications. Deep Convolutional Neural Networks (deep CNNs) have lately been employed with remarkable success to decode EEG/ECoG signals. However, the optimal architectural/training parameter values in these deep CNN architectures have received little attention. In addition, new data-driven optimization methodologies that leverage significant advancements in Machine Learning, such as the Transformer model, have recently been proposed. Because an exhaustive search on all possible architectural/training parameter values of the state-of-the-art deep CNN model (our baseline model) decoding the motor imagery EEG and finger tension ECoG signals comprising the BCI IV 2a and 4 datasets, respectively, would require prohibitively much time, this paper proposes a model-based optimization technique based on the Transformer model for the discovery of the optimal architectural/training parameter values for that model. Our findings indicate that we could pick better values for the architectural/training parameters of the baseline model, enhancing the accuracy of the baseline model by 3.4% in the BCI IV 2a dataset and by 29.8% in the BCI IV 4 dataset.