By leveraging the superposition property, over-the-air computation (OAC) of waveforms enables computations to be performed in an analog fashion in wireless environments, leading to faster computation, lower latency, and reduced energy consumption. In this paper, we investigate reconfigurable intelligent surface (RIS)-aided multiple-input-multiple-output (MIMO) OAC systems designed to emulate the fully-connected (FC) layer of a neural network (NN) via analog OAC, where the RIS and the transceivers are jointly adjusted to engineer the ambient wireless propagation environment to emulate the weights of the target FC layer. We refer to this novel computational paradigm asAirFC. We first study the case in which the precoder, combiner, and RIS phase shift matrices are jointly optimized to minimize the mismatch between the OAC system and the target FC layer. To solve this non-convex optimization problem, we propose a low-complexity alternating optimization algorithm, where semi-closed-form/closed-form solutions for all optimization variables are derived. Next, we consider training of the system parameters using two distinct learning strategies, namelycentralized traininganddistributed training. In the centralized training approach, training is performed at either the transmitter or the receiver, whichever possesses the channel state information (CSI), and the trained parameters are provided to the other terminal. In the distributed training approach, the transmitter and receiver iteratively update their parameters through back and forth transmissions by leveraging channel reciprocity, thereby avoiding CSI acquisition and significantly reducing computational complexity. Subsequently, we extend our analysis to a multi-RIS scenario by exploiting its spatial diversity gain to enhance the system performance, i.e., classification accuracy. Simulation results show that the AirFC system realized by the RIS-aided MIMO configuration achieves satisfactory classification accuracy. Furthermore, it is shown that the multi-RIS system brings significant improvement in terms of the classification accuracy, especially in line-of-sight (LoS)-dominated wireless environments.

Large language models have demonstrated the ability to performin-context learning(ICL), whereby the model performs predictions by directly mapping the query and a few examples from the given task to the output variable. In this paper, we study ICL for deep joint source-channel coding (DeepJSCC) in image transmission over multiple-input multiple-output (MIMO) systems, where an ICL denoiser is employed for MIMO symbol estimation. We first study the transceiver without any hardware impairments and explore the integration of transformer-based ICL with DeepJSCC in both open-loop and closed-loop MIMO systems, depending on the availability of channel state information (CSI) at the transceiver. For both open-loop and closed-loop scenarios, we propose two MIMO transceiver architectures that leverage context information, i.e., pilot sequences and their outputs, as additional inputs, enabling the DeepJSCC encoder, DeepJSCC decoder, and the ICL denoiser to jointly learn encoding, decoding, and estimation strategies tailored to each channel realization. Next, we extend our study to a more challenging scenario where the transceiver suffers from in-phase and quadrature (IQ) imbalance, resulting in nonlinear MIMO estimation. In this case, the context information is also exploited, facilitating joint learning across the DeepJSCC encoder, decoder, and the ICL denoiser under hardware impairments and varying channel conditions. Numerical results demonstrate that the ICL denoiser for MIMO estimation significantly outperforms the conventional least-squares method, with even greater advantages under IQ imbalance. Moreover, the proposed transformer-based ICL framework, integrated with contextual information, achieves significant improvements in end-to-end image reconstruction quality under transceiver IQ imbalance.

We propose a novel deep learning based method to design a coded waveform for integrated sensing and communication (ISAC) system based on orthogonal frequency-division multiplexing (OFDM). Our ultimate goal is to design a coded waveform, which is capable of providing satisfactory sensing performance of the target while maintaining high communication quality measured in terms of the bit error rate (BER). The proposed LISAC provides an improved waveform design with the assistance of deep neural networks for the encoding and decoding of the information bits. In particular, the transmitter, parameterized by a recurrent neural network (RNN), encodes the input bit sequence into the transmitted waveform for both sensing and communications. The receiver employs a RNN-based decoder to decode the information bits while the transmitter senses the target via maximum likelihood detection. We optimize the system considering both the communication and sensing performance. Simulation results show that the proposed LISAC waveform achieves a better tradeoff curve compared to existing alternatives.

Efficient data transmission across mobile multi-hop networks that connect edge devices to core servers presents significant challenges, particularly due to the variability in link qualities between wireless and wired segments. This variability necessitates a robust transmission scheme that transcends the limitations of existing deep joint source-channel coding (Deep-JSCC) strategies, which often struggle at the intersection of analog and digital methods. Addressing this need, this paper introduces a novel hybrid DeepJSCC framework, h-DJSCC, tailored for effective image transmission from edge devices through a network architecture that includes initial wireless transmission followed by multiple wired hops. Our approach harnesses the strengths of DeepJSCC for the initial, variable-quality wireless link to avoid the cliff effect inherent in purely digital schemes. For the subsequent wired hops, which feature more stable and high-capacity connections, we implement digital compression and forwarding techniques to prevent noise accumulation. This dual-mode strategy is adaptable even in scenarios with limited knowledge of the image distribution, enhancing the framework’s robustness and utility. Extensive numerical simulations demonstrate that our hybrid solution outperforms traditional fully digital approaches by effectively managing transitions between different network segments and optimizing for variable signal-to-noise ratios (SNRs). We also introduce a fully adaptive h-DJSCC architecture with both SNR-adaptive (SA) and rate-adaptive (RA) modules capable of adjusting to different network conditions and achieving diverse rate-distortion objectives, thereby reducing the memory requirements on network nodes.

3D point cloud is a three-dimensional data format generated by LiDARs and depth sensors, and is being increasingly used in a large variety of applications from autonomous vehicles to robotics and metaverse. This paper presents novel solutions for the efficient and reliable transmission of point clouds over wireless channels for real-time applications. We first propose SEmatic Point cloud Transmission (SEPT) for small-scale point clouds, which encodes the point cloud via an iterative downsampling and feature extraction process. At the receiver, SEPT decoder reconstructs the point cloud with latent reconstruction and offset-based upsampling. A novel channel-adaptive module is proposed to allow SEPT to operate effectively over a wide range of channel conditions. Next, we propose OTA-NeRF, a scheme inspired by neural radiance fields. OTA-NeRF performs voxelization to the point cloud input and learns to encode the voxelized point cloud into a neural network. Instead of transmitting the extracted feature vectors as in SEPT, it transmits the learned neural network weights over the air in an analog fashion along with few hyperparameters that are transmitted digitally. At the receiver, the OTA-NeRF decoder reconstructs the original point cloud using the received noisy neural network weights. To further increase the bandwidth efficiency of the OTA-NeRF scheme, a fine-tuning algorithm is developed, where only a fraction of the neural network weights are retrained and transmitted. Noticing the poor generality of the OTA-NeRF schemes where the neural network weights are trained for a specific point cloud, we propose an alternative approach, termed OTA-MetaNeRF, which encodes different input point clouds into the latent vectors with shared neural network weights. Extensive numerical experiments confirm that the proposed SEPT, OTA-NeRF and OTA-MetaNeRF schemes achieve superior or comparable performance over the conventional approaches, where an octree-based or a learning-based point cloud compression scheme is concatenated with a channel code. As an additional advantage, all schemes mitigate the cliff and leveling effects making them particularly attractive for highly mobile scenarios. Finally, the run-time complexities of the schemes are evaluated to verify the capability of the proposed schemes for real-time communications.

We introduce deep joint source-channel coding (DeepJSCC) schemes for image transmission over cooperative relay channels. The relay either amplifies-and-forwards its received signal, called DeepJSCC-AF, or leverages neural networks to extract relevant features from its received signal, called DeepJSCC-PF (Process-and-Forward). We consider both half- and full-duplex relays, and propose a novel transformer-based model at the relay. For a half-duplex relay, it is shown that the proposed scheme learns to generate correlated signals at the relay and source to obtain beamforming gains. In the full-duplex case, we introduce a novel block-based transmission strategy, in which the source transmits in blocks, and the relay updates its knowledge about the input signal after each block and generates its own signal. To enhance practicality, a single transformer-based model is used at the relay at each block, together with an adaptive transmission module, which allows the model to seamlessly adapt to different channel qualities and the transmission powers. Simulation results demonstrate the superior performance of DeepJSCC-PF compared to the state-of-the-art BPG image compression algorithm operating at the maximum achievable rate of conventional decode-and-forward and compress-and-forward protocols, in both half- and full-duplex relay scenarios over AWGN and Rayleigh fading channels.

We propose a novel hybrid joint source-channel coding (JSCC) scheme for robust image transmission over multihop networks. In the considered scenario, a mobile user wants to deliver an image to its destination over a mobile cellular network. We assume a practical setting, where the links between the nodes belonging to the mobile core network are stable and of high quality, while the link between the mobile user and the first node (e.g., the access point) is potentially time-varying with poorer quality. In recent years, neural network based JSCC schemes (called DeepJSCC) have emerged as promising solutions to overcome the limitations of separation-based fully digital schemes. However, relying on analog transmission, DeepJSCC suffers from noise accumulation over multi-hop networks. Moreover, most of the hops within the mobile core network may be high-capacity wireless connections, calling for digital approaches. To this end, we propose a hybrid solution, where DeepJSCC is adopted for the first hop, while the received signal at the first relay is digitally compressed and forwarded through the mobile core network. We show through numerical simulations that the proposed scheme is able to outperform both the fully analog and fully digital schemes. Thanks to DeepJSCC it can avoid the cliff effect over the first hop, while also avoiding noise forwarding over the mobile core network thank to digital transmission. We believe this work paves the way for the practical deployment of DeepJSCC solutions in 6G and future wireless networks.

We introduce a vision transformer (ViT)-based deep joint source and channel coding (DeepJSCC) scheme for wireless image transmission over multiple-input multiple-output (MIMO) channels, called DeepJSCC-MIMO. We employ DeepJSCC-MIMO in both open-loop and closed-loop MIMO systems. The novel DeepJSCC-MIMO architecture surpasses the classical separation-based benchmarks, while exhibiting robustness to channel estimation errors and flexibility in adapting to diverse channel conditions and antenna configurations without requiring retraining. Specifically, by harnessing the self-attention mechanism of the ViT, DeepJSCC-MIMO intelligently learns feature mapping and power allocation strategies tailored to the unique characteristics of the source image and prevailing channel conditions. Extensive numerical experiments validate the significant improvements in both distortion quality and perceptual quality achieved by DeepJSCC-MIMO for both open-loop and closed-loop MIMO systems across a wide range of scenarios. Moreover, DeepJSCC-MIMO exhibits robustness to varying channel conditions, channel estimation errors, and different antenna numbers, making it an appealing technology for emerging semantic communication systems.

This paper presents a novel vision transformer (ViT) based deep joint source channel coding (DeepJSCC) scheme, dubbed DeepJSCC-l++, which can adapt to different target bandwidth ratios as well as channel signal-to-noise ratios (SNRs) using a single model. To achieve this, we treat the bandwidth ratio and the SNR as channel state information available to the encoder and decoder, which are fed to the model as side information, and train the proposed DeepJSCC-l++ model with different bandwidth ratios and SNRs. The reconstruction losses corresponding to different bandwidth ratios are calculated, and a novel training methodology, which dynamically assigns different weights to the losses of different bandwidth ratios according to their individual reconstruction qualities, is introduced. Shifted window (Swin) transformer is adopted as the backbone for our DeepJSCC-l++ model, and it is shown through extensive simulations that the proposed DeepJSCC-l++ can adapt to different bandwidth ratios and channel SNRs with marginal performance loss compared to the separately trained models. We also observe the proposed schemes can outperform the digital baseline, which concatenates the BPG compression with capacity-achieving channel code. We believe this is an important step towards the implementation of DeepJSCC in practice as a single pre-trained model is sufficient to serve the user in a wide range of channel conditions.

This paper presents a vision transformer (ViT) based joint source and channel coding (JSCC) scheme for wireless image transmission over multiple-input multiple-output (MIMO) systems, called ViT-MIMO. The proposed ViT-MIMO architecture, in addition to outperforming separation-based benchmarks, can flexibly adapt to different channel conditions without requiring retraining. Specifically, exploiting the self-attention mechanism of the ViT enables the proposed ViT-MIMO model to adaptively learn the feature mapping and power allocation based on the source image and channel conditions. Numerical experiments show that ViT-MIMO can significantly improve the transmission quality across a large variety of scenarios, including varying channel conditions, making it an attractive solution for emerging semantic communication systems.

Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning-based near-orthogonal superposition (NOS) coding scheme is proposed to transmit short messages in multiple-input multiple-output (MIMO) channels for mMTC applications. In the proposed MIMO-NOS scheme, a neural network-based encoder is optimized via end-to-end learning with a corresponding neural network-based detector/decoder in a superposition-based auto-encoder framework including a MIMO channel. The proposed MIMO-NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector and reshaped for the space-time transmission. For the receiver, we propose a novel looped$K$-best tree-search algorithm with cyclic redundancy check (CRC) assistance to enhance the error correcting ability in the block-fading MIMO channel. For a comprehensive understanding of the proposed MIMO-NOS scheme, we further quantify the gain from individual components/modules in the framework, and analyze the decoding complexity measured by the floating point operations (FLOPs). Simulation results show the proposed MIMO-NOS scheme outperforms maximum likelihood (ML) MIMO detection combined with a polar code with CRC-assisted list decoding by 1 – 2 dB in various MIMO systems for short (32 – 64 bit) message transmission.

Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning based near-orthogonal superposition (NOS) coding scheme is proposed for reliable transmission of short messages in the additive white Gaussian noise (AWGN) channel for mMTC applications. Similar to recent hyper-dimensional modulation (HDM), the NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector for transmission. The NOS decoder first estimates the information vectors and then performs a cyclic redundancy check (CRC)-assisted K-best tree-search algorithm to further reduce the packet error rate. The proposed NOS encoder and decoder are deep neural networks (DNNs) jointly trained as an auto-encoder and decoder pair to learn a new NOS coding scheme with near-orthogonal codewords. Simulation results show the proposed deep learning-based NOS scheme outperforms HDM and Polar code with CRC-aided list decoding for short (32-bit) message transmission.

We present a deep learning based joint source channel coding (JSCC) scheme for wireless image transmission over multipath fading channels with non-linear signal clipping. The proposed encoder and decoder use convolutional neural networks (CNN) and directly map the source images to complex-valued baseband samples for orthogonal frequency division multiplexing (OFDM) transmission. The proposed model-driven machine learning approach eliminates the need for separate source and channel coding while integrating an OFDM datapath to cope with multipath fading channels. The end-to-end JSCC communication system combines trainable CNN layers with non-trainable but differentiable layers representing the multipath channel model and OFDM signal processing blocks. Our results show that injecting domain expert knowledge by incorporating OFDM baseband processing blocks into the machine learning framework significantly enhances the overall performance compared to an unstructured CNN. Our method outperforms conventional schemes that employ state-of-the-art but separate source and channel coding such as BPG and LDPC with OFDM. Moreover, our method is shown to be robust against non-linear signal clipping in OFDM for various channel conditions that do not match the model parameter used during the training.

In order to reduce the downlink training overhead of mmWave communications, we propose a novel downlink beamforming strategy using the uplink sub-6GHz channel and downlink mmWave pilots that are sent from a few active antennas. Specifically, we design a novel dual-input neural network architecture, called FusionNet, to merge the sub-6GHz channel and the channel of a few active mmWave antennas. The proposed fusion model could intelligently adjust the attention paid (by the neural network) for sub-6GHz channel and mmWave channel by an attention mechanism. The output of the FusionNet represents the probability for each beam being the optimal one. We also propose an antenna selection model that can choose better active antennas to send the downlink pilots, in which the gradient of antenna selection vector is approximated by that of an antenna probability vector. Simulation results demonstrate the superior performance of the proposed strategy compared to the existing one that purely relies on the sub-6GHz information or compared to the shallow model that directly adds uniform pilots.