In this work, we propose a novel integrated sensing and communication (ISAC) framework for connected and autonomous vehicles (CAVs), which incorporates digital semantic communication (SemCom) to achieve both reliable communication and accurate sensing. Within this framework, the transmitting vehicle extracts semantic symbols from the source data and transmits them over orthogonal frequency division multiplexing (OFDM) sub-carriers, while simultaneously utilizing echo signals for radar-based environmental sensing. To achieve reliable SemCom, the transmitter must jointly optimize the quantization bitwidth for semantic symbols, the modulation order, the power allocation across semantic symbol dimensions, and the transmit beamforming strategy. These optimizations must also consider sensing performance, leading to a tradeoff between radar sensing and task-oriented SemCom. To address this joint optimization problem, we decompose it into three subproblems and develop corresponding solutions: 1) a hierarchical constrained proximal policy optimization (H-CPPO) algorithm to determine the quantization bitwidth, modulation order, and power allocation under frequency-flat channels, 2) a joint beamforming strategy to optimize the dual-function radar-SemCom transmit beamforming vector, and 3) a semantic importance-based signal-to-noise ratio (SNR) matching strategy that effectively adapts the optimal power allocation obtained under frequency-flat conditions to fading channels with random gains. Simulation results on a road image segmentation task show that, the proposed SemCom scheme achieves near-optimal segmentation accuracy while reducing radar beamforming error by up to 82% compared to the conventional digital system using the same quadrature phase shift keying (QPSK) modulation.

This paper proposes a novel integrated sensing and communication (ISAC) framework incorporating digital semantic communication (SemCom) to resolve the tradeoff between sensing and communication performance. In particular, to accomplish task-oriented semCom, the base station (BS) extracts semantic symbols and transmits each dimension over different orthogonal frequency division multiplex (OFDM) subcarriers. To achieve sensing objective, the BS broadcasts OFDM signals and receives echoes via a uniform linear array (ULA) to estimate echo channel state information (CSI) and obtain target parameters. Given the varying task-related importance of each dimension, the framework allocates quantization bits, modulation order, and transmission power accordingly to meet SemCom requirements. On the other hand, sensing performance is evaluated using the Cramér-Rao Bound (CRB) of echo CSI, with transmission power allocation optimized to enhance sensing. The problem is formulated to minimize the sensing CRB while satisfying SemCom task loss, total resources, and transmission efficiency constraints. To solve this problem, we introduce a Hybrid Action Space Proximal Policy Optimization (H-PPO) algorithm, which can simultaneously determine the power allocated for each dimension from a continuous action space, and select a proper number of quantization bits and modulation order from discrete action spaces. Simulations show that the proposed method enhances SemCom task performance by up to 77% and reduces sensing error by up to 58% compared to conventional digital systems.

In this paper, a multi-modal data based semi-supervised learning (SSL) framework that jointly use channel state information (CSI) data and RGB images for vehicle positioning is designed. In particular, an outdoor positioning system where the vehicle locations are determined by a base station (BS) is considered. The BS equipped with several cameras can collect a large amount of unlabeled CSI data and a small number of labeled CSI data of vehicles, and the images taken by cameras. Although the collected images contain partial information of vehicles (i.e. azimuth angles of vehicles), the relationship between the unlabeled CSI data and its azimuth angle, and the distances between the BS and the vehicles captured by images are both unknown. Therefore, the images cannot be directly used as the labels of unlabeled CSI data to train a positioning model. To exploit unlabeled CSI data and images, a SSL framework that consists of a pretraining stage and a downstream training stage is proposed. In the pretraining stage, the azimuth angles obtained from the images are considered as the labels of unlabeled CSI data to pretrain the positioning model. In the downstream training stage, a small sized labeled dataset in which the accurate vehicle positions are considered as labels is used to retrain the model. Simulation results show that the proposed method can reduce the positioning error by up to 30% compared to a baseline where the model is not pretrained.

In this paper, a multi-modal vehicle positioning framework that jointly localizes vehicles with channel state information (CSI) and images is designed. In particular, we consider an outdoor scenario where each vehicle can communicate with only one BS, and hence, it can upload its estimated CSI to only its associated BS. Each BS is equipped with a set of cameras, such that it can collect a small number of labeled CSI, a large number of unlabeled CSI, and the images taken by cameras. To exploit the unlabeled CSI data and position labels obtained from images, we design an meta-learning based hard expectation-maximization (EM) algorithm. Specifically, since we do not know the corresponding relationship between unlabeled CSI and the multiple vehicle locations in images, we formulate the calculation of the training objective as a minimum matching problem. To reduce the impact of label noises caused by incorrect matching between unlabeled CSI and vehicle locations obtained from images and achieve better convergence, we introduce a weighted loss function on the unlabeled datasets, and study the use of a meta-learning algorithm for computing the weighted loss. Subsequently, the model parameters are updated according to the weighted loss function of unlabeled CSI samples and their matched position labels obtained from images. Simulation results show that the proposed method can reduce the positioning error by up to 61% compared to a baseline that does not use images and uses only CSI fingerprint for vehicle positioning.

In this paper, a multi-modal vehicle positioning framework that jointly localizes vehicles with channel state information (CSI) and images is designed. In particular, we consider an outdoor scenario where each vehicle can communicate with only one base station (BS), and hence, it can upload its estimated CSI to only its associated BS. Each BS is equipped with a set of cameras, such that it can collect a small number of labeled CSI, a large number of unlabeled CSI, and the images taken by cameras. To exploit the unlabeled CSI data and position labels obtained from images, we design a hard expectation-maximization (EM) based deep learning (DL) algorithm. Specifically, since we do not know the corresponding relationship between unlabeled CSI and the multiple vehicle locations in images, we formulate the calculation of the log-likelihood function as a maximum matching problem. Subsequently, the model parameters are updated according to the maximum matching between unlabeled CSI and position labels obtained from images. Simulation results show that the proposed method can reduce the positioning error by up to 60% compared to a baseline that does not use images and uses only CSI fingerprint for vehicle positioning.