Foundation deep learning (DL) models are general models, designed to learn general, robust and adaptable representations of their target modality, enabling finetuning across a range of downstream tasks. These models are pretrained on large, unlabeled datasets using self-supervised learning (SSL). Foundation models have demonstrated better generalization than traditional supervised approaches, a critical requirement for wireless communications where the dynamic environment demands model adaptability. In this work, we propose and demonstrate the effectiveness of a Vision Transformer (ViT) as a radio foundation model for spectrogram learning. We introduce a Masked Spectrogram Modeling (MSM) approach to pretrain the ViT in a selfsupervised fashion. We evaluate the ViT-based foundation model on two downstream tasks: Human Activity sensing and Spectrogram Segmentation. Experimental results demonstrate competitive performance to supervised training while generalizing across diverse domains. Notably, the pretrained ViT model outperforms a four-times larger model that is trained from scratch on the spectrogram segmentation task, while requiring significantly less training time, and achieves competitive performance on the human activity sensing task. This work demonstrates the effectiveness of ViT with MSM for pretraining as a promising technique for scalable foundation model development in future 6G networks.

The deployment of deep learning (DL) models on Internet of Things (IoT) devices is constrained by limited computational resources, necessitating effective model compression techniques. This paper introduces a novel, structured, nonuniform, block-wise compression approach, tailored for resourceconstrained IoT applications that require Angle of Arrival (AoA) estimation. Our method prunes all model blocks of layers, adjusting the pruning intensity based on each block's measured contribution to overall performance: less critical blocks undergo more aggressive pruning, while essential blocks are preserved to maintain accuracy. Applied to MobileNetV3 for AoA estimation, our technique achieved a 35.9X reduction in parameter count, an 11.5X reduction in model size, a 7.7X reduction in multiplyaccumulate operations (MACs), a 4.7X improvement in inference speed on CPU, and a 2.4X improvement on GPU, with minimal impact on accuracy. These results are achieved on a real-world dataset collected on a software defined radio (SDR) testbed to validate the effectiveness of the proposed solution. This demonstrates that our approach can achieve a highly favorable trade-off between compression and performance, supporting the deployment of tiny AoA models in latency-critical and powersensitive IoT environments.

Artificial intelligence (AI) is anticipated to play a pivotal role in 6G. However, a key challenge in developing AIpowered solutions is the extensive data collection and labeling efforts required to train supervised deep learning models. To overcome this, self-supervised learning (SSL) approaches have recently demonstrated remarkable success across various domains by leveraging large volumes of unlabeled data to achieve nearsupervised performance. In this paper, we propose an effective SSL scheme for radio signal representation learning using momentum contrast. By applying contrastive learning, our method extracts robust, transferable representations from a large realworld dataset. We assess the generalizability of these learned representations across two wireless communications tasks: angle of arrival (AOA) estimation and automatic modulation classification (AMC). Our results show that carefully designed augmentations and diverse data enable contrastive learning to produce highquality, invariant latent representations. These representations are effective even with frozen encoder weights, and fine-tuning further enhances performance, surpassing supervised baselines. To the best of our knowledge, this is the first work to propose and demonstrate the effectiveness of self-supervised learning for radio signals across multiple tasks. Our findings highlight the potential of self-supervised learning to transform AI for wireless communications by reducing dependence on labeled data and improving model generalization - paving the way for scalable foundational 6G AI models and solutions.

The Internet of Things (IoT) integrates deep learning (DL) to enhance real-time data processing across diverse applications. However, deploying DL models on resourceconstrained IoT devices remains challenging, especially for tasks such as Angle-of-Arrival (AoA) estimation in dynamic environments. Variations in deployment conditions, such as changing modulation schemes, lead to domain shifts that degrade traditional models' performance, underscoring the need for adaptive, low-complexity DL frameworks. This paper introduces a novel compact phase and amplitude representation within a Prototypical Network-based approach, optimized for domain-adaptive AoA prediction in IoT and validated using real data from a softwaredefined radio (SDR) testbed. Compared to covariance and raw IQ data, our proposed representation reduces Mean Absolute Error (MAE) by approximately 32 % and 55 %, respectively, in unseen modulation scenarios. Further, evaluations on an SDR dataset collected using a$2 \times 2$Uniform Rectangular Array (URA) configuration with seven modulation schemes demonstrate that Prototypical Networks with few-shot learning enable accurate and robust adaptation with minimal data, maintaining high accuracy across both seen and unseen modulations

Foundational deep learning (DL) models are general models, trained on large, diverse, and unlabelled datasets, typically using self-supervised learning techniques - and have led to significant advancements especially in natural language processing. These pretrained models can be fine-tuned for related downstream tasks, offering faster development and reduced training costs, while often achieving improved performance. In this work, we introduce Masked Spectrogram Modeling, a novel self-supervised learning approach for pretraining foundational DL models on radio signals. Adopting a Convolutional LSTM architecture for efficient spatio-temporal processing, we pretrain the model with an unlabelled radio dataset collected from over-the-air measurements. Subsequently, the pretrained model is fine-tuned for two downstream tasks: spectrum forecasting and segmentation. Experimental results demonstrate that our methodology achieves competitive performance in both forecasting accuracy and segmentation, validating its effectiveness for developing foundational radio models.

Deep learning (DL) techniques have recently emerged to efficiently manage mmWave beam transmissions without requiring time consuming beam sweeping strategies. A fundamental challenge in these methods is their dependency on hardware-specific training data and their limited ability to generalize. Large drops in performance are reported in literature when DL models trained in one antenna environment are applied in another. This paper proposes the application of Prototypical Networks to address this challenge – and utilizes the DeepBeam real-world dataset [1] to validate the developed solutions. Prototypical Networks (PN) excel in extracting features to establish class-specific prototypes during the training, resulting in precise embeddings that encapsulate the defining features of the data. We demonstrate the effectiveness of PN to enable generalization of deep beam predictors across unseen antennas. Our approach, which integrates data normalization and prototype normalization with the PN, achieves an average beam classification accuracy of 74.11% when trained and tested on different antenna datasets. This is an improvement of 398% compared to baseline performances reported in literature that do not account for such domain shifts. To the best of our knowledge, this work represents the first demonstration of the value of Prototypical Networks for domain adaptation in wireless networks, providing a foundation for future research in this area.