A pinching-antenna system (PASS)-enhanced mobile edge computing (MEC) architecture is investigated to improve the task offloading efficiency and latency performance in dynamic wireless environments. By leveraging dielectric waveguides and flexibly adjustable pinching antennas, PASS establishes short-distance line-of-sight (LoS) links while effectively mitigating the significant path loss and potential signal blockage, making it a promising solution for high-frequency MEC systems. We formulate a network latency minimization problem to joint optimize uplink PASS beamforming and task offloading. The resulting problem is modeled as a Markov decision process (MDP) and solved via the deep reinforcement learning (DRL) method. To address the instability introduced by the max operator in the objective function, we propose a load balancing-aware proximal policy optimization (LBPPO) algorithm. LBPPO incorporates both node-level and waveguide-level load balancing information into the policy design, maintaining computational and transmission delay equilibrium, respectively. Simulation results demonstrate that the proposed PASS-enhanced MEC with adaptive uplink PASS beamforming exhibit stronger convergence capability than fixed-PA baselines and conventional MIMO-assisted MEC, especially in scenarios with a large number of UEs or high transmit power.

Multiple simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) aided cell-free massive multiple-input multiple-output (CF mMIMO) network is investigated. A long-term sum-rate maximization problem is formulated to jointly optimize the active beamforming at each access point (AP) and the passive beamforming at each STAR-RIS while satisfying the quality of service (QoS) requirements. To address this non-convex problem with challenges caused by user mobility and high-dimensional optimization variables, two deep reinforcement learning (DRL)-based beamforming algorithms are proposed. Firstly, a soft actor-critic (SAC)-based centralized joint beamforming algorithm is proposed, which adds maximum entropy term to the objective function and provides an exceptional exploration-exploitation trade-off. However, since the centralized scheme might suffer from high communication loads and latency, we generalize the proposed approach into a distributed control. A multi-agent SAC (MASAC)-based distributed beamforming algorithm is further proposed, which employs the centralized training and decentralized execution (CTDE) framework, where each AP plays a role of an agent and makes decisions based on local observations. Simulation results demonstrate that: 1) The multiple STAR-RISs can effectively improve the performance of the CF mMIMO networks compared to other baseline schemes; 2) Both SAC-based and MASAC-based beamforming algorithms can maximize the sum-rate and guarantee the QoS of users in the long term; and 3) The MASAC-based beamforming algorithm performs better and converges faster than the SAC-based beamforming algorithm as it reduces the overall complexity and alleviates pressure on the fronthaul link transmission by making decentralized decisions.

Constructing radio maps traditionally requires extensive site surveys with precise location labels, resulting in costly and time-consuming calibration. Conventional approaches derive labels from inertial measurement units (IMUs), but are constrained by device-level access permissions and the need for pre-installed IMU hardware. In this paper, we present a calibration-free radio map construction method that relies solely on Channel State Information (CSI) measurements, thereby obviating the need for location labels. Our key insight is to embed CSI data into a two-dimensional geographic space using a neural network, without any label information. The primary challenge is to preserve the real-world topology in the embedded space. To address this issue, we propose a novelTopology-Guided Manifold Learning (TGML)approach that learns a low-dimensional embedding through self-supervision based on its mapping to a topological map in the geographic space. Specifically, we introduce an embedding network that learns locally smoothed proximity, thus creating a compact low-dimensional representation in the latent space. We further employ a regularized transport method with a differentiable Sinkhorn distance to establish an optimal mapping between the latent and geographic spaces. We use the resulting mapping deviation to supervise the training of the embedding network, enabling continuous refinement through self-supervision. Experiments in an office environment show that TGML achieves an average localization error of 2.37m and a relative beam estimation error of 6.68%, outperforming state-of-the-art methods.

Due to the increasing number of users and antennas in extremely large antenna arrays (ELAA) based integrated sensing and communication (ISAC) systems, the complexity of beamforming optimization becomes overwhelming, which impedes real-time and cost-efficient ISAC deployment in practice. Specifically, a general ISAC system where the base station (BS) communicates with multiple users and performs target detection is considered. Then, a sum communication rate maximization problem is formulated, subjected to the constraints of transmit power and the minimum sensing rates of users. To solve this problem, we develop a framework that leverages deep learning algorithms to provide a low complexity and transferable (LCT) solution for ISAC beamforming. The proposed LCT beamforming optimization framework includes three modules: 1) an unsupervised learning based feature extraction algorithm is proposed to extract fixed-size latent features while keeping its essential information from the variable channel state information (CSI); 2) a reinforcement learning (RL) based beampattern optimization algorithm is proposed to search the desired beampattern according to the extracted features; 3) a supervised learning based beamforming reconstruction algorithm is proposed to reconstruct the beamforming vector from beampattern given by the RL agent. Simulation results demonstrate that the proposed LCT framework outperforms the baseline RL algorithm by optimizing the intuitional beampattern rather than beamforming. Moreover, the LCT framework provides a solution for low-cost beamforming optimization in ISAC systems. The trained RL module can be transferred without retraining when the antenna or user number changes.

In this work, a multiple waveguide pinching antenna system (PASS) assisted integrated sensing and communication (ISAC) system is proposed, where the base station (BS) is equipped with transmitting pinching antennas (PAs) and receiving uniform linear array (ULA) antennas. The PASS-transmitting- ULA-receiving (PTUR) BS transmits the communication and sensing signals through the PAs on waveguides and collects the echo sensing signals with the mounted ULA. Based on this configuration, a target sensing Cramèr–Rao Bound (CRB) minimization problem is formulated under communication quality-of-service (QoS) constraints, power budget constraint, and PA deployment constraints. To tackle the resulting non-convex problem, an alternating optimization (AO) framework is developed, which decomposes the problem into a digital beamforming sub-problem and a pinching beamforming sub-problem. The digital beamforming design is optimized via semidefinite relaxation (SDR), while the PA deployment is updated using penalty-based method. Simulation results demonstrate that: 1) the proposed PASS assisted ISAC framework achieves superior performance over benchmark schemes; and 2) the PASS assisted ISAC is less affected by stringent communication constraints compared to conventional MIMO-ISAC, and benefits from increasing the number of waveguides and PAs per waveguide.

A novel mobile edge generation (MEG) framework is proposed to support latency-sensitive generation tasks in the social-aware device-to-device (D2D) underlaying cellular network. Within this framework, user devices (UDs) and base station (BS) are organized into socially cohesive communities based on content preference and spatial proximity, enabling cooperative generation tasks via both cellular and intra-community D2D communications. A joint seed-and-content based BS-D2D (JSCB) transmission protocol is proposed to dynamically orchestrate the transmission mode between seed acquisition with local generation and direct content sharing across multiple consecutive task rounds, incorporating the spillover mechanism for handling overdue transmissions. Based on this protocol, an average task delay minimization problem is formulated to jointly optimize the UD association between cellular and D2D communication, transmission mode, D2D pairing, and BS-side beamforming. To efficiently solve the hybrid and temporally coupled problem, a joint matching and proximal policy optimization (JMPPO) algorithm is developed, where the discrete and continuous actions are decoupled with specialized modules though a hierarchical deep reinforcement learning and matching design. Numerical results validate that 1) the JSCB protocol reduces delay through adaptive transmission scheduling and cellular/D2D coordination; 2) the JMPPO algorithm outperforms both learning-based and traditional baselines in terms of average delay under the spillover and hybrid action scenarios; 3) the proposed schemes demonstrate robustness across diverse network and system conditions.

In this paper, a general ISAC system where the base station (BS) communicates with multiple users and performs target detection is considered. Then, a sum communication rate maximization problem is formulated, subjected to the constraints of transmit power and the minimum sensing rates of users. To solve this problem, we develop a framework that leverages deep learning algorithms to provide a three-stage solution for ISAC beamforming. The three-stage beamforming optimization solution includes three modules: 1) an unsupervised learning based feature extraction algorithm is proposed to extract fixed-size latent features while keeping its essential information from the variable channel state information (CSI); 2) a reinforcement learning (RL) based beampattern optimization algorithm is proposed to search the desired beampattern according to the extracted features; 3) a supervised learning based beamforming reconstruction algorithm is proposed to reconstruct the beamforming vector from beampattern given by the RL agent. Simulation results demonstrate that the proposed three-stage solution outperforms the baseline RL algorithm by optimizing the intuitional beampattern rather than beamforming.

Obtaining data on active travel activities such as walking, jogging, and cycling are important for refining sustainable transportation systems (STS). In order to provide an accurate and privacy-preserving sensing solution, a deep learning (DL)-enhanced distributed acoustic sensing (DAS) system for recognizing active travel activities is proposed. By leveraging the ambient vibrations captured by DAS, this scheme infers motion patterns without relying on image-based or wearable devices, thereby addressing privacy concerns. We conduct real-world experiments in two geographically distinct locations and collect a comprehensive dataset to evaluate the performance of the proposed system. To address the generalization challenges posed by heterogeneous deployment environments, we propose two solutions based on network availability: 1) an Internet-of-Things (IoT) scheme based on federated learning (FL) is proposed, and it enables geographically different DAS nodes to be trained collaboratively to improve generality; 2) an off-line initialization approach enabled by meta-learning is proposed to develop high-generality initialization for DL models and to enable rapid model fine-tuning with limited data samples, facilitating generalization in newly established or isolated DAS nodes. Experimental results of the walking and cycling classification problem demonstrate the performance and generality of the DL-enhanced DAS system, paving the way for practical, large-scale DAS monitoring of active travel.

The Internet of Things (IoT) introduces diverse requirements and ubiquitous connections, necessitating efficient and affordable energy consumption as the ecosystem continues to grow. To address this challenge, we investigate a pure nonorthogonal multiple access (pure-NOMA) beamforming scheme to enhance system capacity by accommodating more IoT devices within the same spectrum. An energy efficiency (EE) maximization problem is formulated, jointly optimizing the beamforming matrix, power allocation, and device clustering. Due to the dynamic nature of the transmission channel and the coupling nonconvex mixed integer nonlinear programming (MINLP) problem, it is challenging to solve this problem by conventional mathematical methods. Additionally, the high dimensionality and coupling nonconvex MINLP problem pose significant challenges for traditional reinforcement learning (RL) methods. To overcome these issues, we propose a curiosity-driven approach that leverages intrinsic information from the base station (BS) to achieve energy efficient resource allocation. Simulation results demonstrate that pure-NOMA offers up to a 25% improvement in EE compared to hybrid-NOMA, while the curiosity-driven learning method outperforms baseline techniques, including deep RL (DRL), zero-forcing, and random methods, achieving a 14.78% reward gain over the DRL approach. The effectiveness of the proposed method is validated across various beam settings, device counts, quality-of-service requirements, and time consumption metrics, all while maintaining comparable computational complexity.

A distributed acoustic sensor (DAS)-enabled smart vehicle monitoring system is investigated in this article to detect the type and passenger occupancy of vehicles for intelligent transportation systems (ITSs). Accurate detection of the number of occupants and the vehicle type is critical for ITS to monitor the occupancy of vehicles, improve vehicle operation efficiency, and achieve intelligent traffic management. We have developed several deep learning (DL) algorithms for occupancy detection and vehicle type discrimination according to the signals provided by DAS. To be more specific, a novel type of basic neural network structure, namely sparse residual (SR) block is proposed, and several DL models are developed for DAS signals based on the basic SR block unit. The proposed DL approaches are tested using a unique dataset collected from a road experiment. The test results indicate that 1) the proposed SR network (SR-Net) and Alex SR (Alex-SR) network can achieve detection accuracy of over 90%; 2) the proposed models exhibit superior convergence, stability, and accuracy than the conventional residual network (ResNet); and 3) the proposed DL solutions have superiority in terms of complexity and model size compared to benchmarks.

Mobile-edge generation (MEG) is an emerging technology that allows the network to meet the challenging traffic load expectations posed by the rise of generative artificial intelligence (GAI). A novel MEG model is proposed for deploying GAI models on edge servers (ESs) and user equipment (UE) to jointly complete text-to-image generation tasks. In the generation task, the ES and UE will cooperatively generate the image according to the text prompt given by the user. To enable the MEG, a pretrained latent diffusion model (LDM) is invoked to generate the latent feature, and an edge-inferencing MEG protocol is employed for data transmission exchange between the ES and the UE. A compression coding technique is proposed for compressing the latent features to produce seeds. Based on the above seed-enabled MEG model, an image quality optimization problem with energy constraint is formulated. The transmitting power of the seed is dynamically optimized by a deep reinforcement learning (DRL) agent over the fading channel. The proposed MEG-enabled text-to-image generation system is evaluated in terms of image quality and transmission overhead. The numerical results indicate that, compared to the conventional centralized generation-and-downloading scheme, the symbol number of the transmission of MEG is materially reduced. In addition, the proposed compression coding approach can improve the quality of generated images under low signal-to-noise ratio (SNR) conditions, and the DRL-enabled dynamic power control further improves the image quality under the energy constraint compared to static transmit power control.

The novel concept of mobile edge generation (MEG) is investigated, where the generative artificial intelligence (GAI) model is partitioned into sub-models to be distributed in the network edge, thus enabling latent feature exchange between the edge server and user equipments (UEs). A seed coding module is introduced to encode the intermediate latent features generated by the GAI sub-model at the edge server into flexibly-sized seed for transmission to UEs, instead of transmitting large-size raw data. A weighted energy consumption minimization problem is formulated by jointly optimizing the seed coding ratio (SCR), transmit power, and computing frequencies while guaranteeing the quality-of-generation requirements including total latency and peak signal-to-noise ratio (PSNR). To enhance the resilience of the MEG models against the channel noise, a joint fine-tuning scheme based on low-rank adaption is proposed to train the introduced rank-reduced bypass matrices and seed coding module. Based on the fine-tuned results, a PSNR model regarding SCR and communication signal-to-noise ratio is established to overcome the optimization difficulty due to the lack of the explicit PSNR model. A proximal policy optimization-based MEG energy consumption optimization (MEG-ECO) algorithm is proposed to solve the formulated problem, where the order of magnitude balancing on state and penalty shaping are exploited for more efficient learning. Numerical results reveal that 1) the fine-tuned MEG models have superior resilience against the channel noise; 2) the proposed MEG-ECO algorithm can significantly reduce energy consumption by up to 87.4% compared to conventional centralized generation and up to 33.5% against MEG without seed coding module; and 3) the energy consumption decreases when more partial models are assigned to the edge server, whereas this impact diminishes as the latency threshold is relaxed.

A multiple unmanned aerial vehicles (UAVs) enabled integrated sensing and communication (ISAC) system is investigated. In contrast to existing UAV-enabled ISAC systems assuming static users or 2D UAV trajectory, we consider a practical roaming user scenario and a 3D deployment for UAVs. Then, a joint trajectory and beamforming optimization problem is formulated for maximizing the long-term sum data rate, subject to the transmitting power constraint and ensuring beam pattern gain constraint for sensing target. We proposed a two-step approach for against the dynamic scenario: 1) a K-means based hierarchical user association algorithm is proposed to renew the user association periodically. 2) a hybrid reward multi-agent proximal policy optimization (HR-MAPPO) algorithm is proposed, which decomposes the complex combined reward into a team reward and an individual reward. Numerical results demonstrate that the proposed HR-MAPPO algorithm can outperform conventional single-agent and multi-agent RL algorithms by maintaining high scores on both sum data rate and beam pattern gain.

An extremely large-scale antenna array (ELAA) based integrated sensing and communication (ISAC) system is investigated. In contrast to existing near-field ISAC systems, we consider the complexity of beamforming design and antenna selection for ELAA under a user-roaming environment. The formulated problem is maximizing long-term sum data rate by jointly optimizing antenna selection and beamforming, subject to the constraints of transmitting power and sensing targets’ beam pattern gain. We propose a dimension reduction assisted multi-agent proximal policy optimization (DR-MAPPO) algorithm against the dynamic scenario: 1) an OrderConv layer is designed, which reduces the input dimension by removing the channel state information of unselected antenna while keeping the order information of selected ones, 2) multiple agents are employed to execute antenna selection and beamforming in a distributed manner. Numerical results demonstrate that the proposed DR-MAPPO algorithm improve the computational efficiency of joint antenna selection and beamforming optimization by reducing the size of the input dimension while obtaining comparable data rates and beampattern gain for sensing targets.

The cell-free massive multiple-input multiple-output (CF mMIMO) network is emerging as an innovative technology that utilizes numerous access points (APs) distributed across a region to provide services for users through coherent transmission and reception. To enhance the coverage of the CF mMIMO network, we introduce simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) into the CF mMIMO network. In the STAR-RIS aided CF mMIMO network, beamforming for the downlink of mobile users becomes exceptionally complex and challenging. Therefore, we propose a beamforming algorithm based on soft actor-critic (SAC), which can jointly optimize the beamforming of APs and phase shifts and amplitude coefficients of STAR-RISs. Simulation results demonstrate that the proposed SAC-based beamforming algorithm significantly maximizes the sum-rate of the STAR-RIS aided CF mMIMO network, outperforming the no STAR-RIS aided CF mMIMO network in terms of meeting quality of service (QoS) requirement.

Mobile edge generation (MEG) is an emerging technology that allows the network to meet the challenging traffic load expectations posed by the rise of generative artificial intelligence (GAI). A novel MEG model is proposed for deploying GAI models on edge servers (ES) and user equipment (UE) to jointly complete test-to-image generation tasks. In the generaation task, the user uploads the text prompt and the ES and UE will cooperatively generate the image for the user. To enable the data transmission exchange between the ES and the UE, a seed based MEG protocol is employed, where a coded latent feature is created as a generation seed. A pre-trained latent diffusion model (LDM) is invoked to generate the latent feature, and a compression coding technique is proposed for compressing the latent features. The proposed MEG enabled text-to-image generation system is evaluated in terms of image quality and transmission overhead. The numerical results indicate that, compared to the conventional centralized generation-and-downloading scheme, the symbol number of the transmission of MEG is materially reduced. In addition, the proposed compression coding approach can improve the quality of generated images under low signal-to-noise ratio (SNR) conditions.

The advent of mobile edge computing (MEC) technology offers new prospects for executing demanding applications close to the user. However, complex applications like intelligent transportation and autonomous driving pose modeling and problem-solving challenges due to inter-user service logic correlations. Therefore, we construct a model that considers the terminal's mobility, time-varying channel status, and inter-user task dependencies and formulate a problem aiming to optimize the task completion delay and the energy consumption weighted cost in a dynamic MEC scenario. To resolve this problem, a Double Deep Q Network (DDQN)-based algorithm is developed for task offloading, while integrated sub channel allocation and transmit power control constitute part of the interaction with the dynamic environment to generate the reward signal, optimizing the long-term system performance. Comprehensive simulations verify that the proposed algorithm outperforms the comparative methods in terms of reducing the cost, and its adaptability in different scenarios has also been validated and analyzed.

Multiple access is one of the primary issues for multi-user semantic communication systems. In this paper, we propose a novel pair of semantic difference (SeD) aware NOMA transceivers for downlink semantic-based image transmission, which mitigates the semantic-level interference among semantic streams. In specific, a SeD-aware superposition coding (SC) technique is proposed to suppress the semantic-level interference by coupling the semantic symbols of higher inter-feature semantic difference, which makes the interfering semantic symbols be identified and filtered out by the corresponding decoding function. The SeD-aware successive interference cancellation (SIC) technique further reduces the semantic-level interference by estimating the transmitted semantic symbols with the joint semantic and channel (JSC) autoencoder. Simulation results show that the proposed transceivers achieve comparable performance with benchmarks of OMA-aided transmission, while outperforming the benchmark of SeD-unaware NOMA transceivers in terms of the quality of reconstructed images and outperforming both benchmarks in terms of semantic transmission efficiency.

A near-field multi-input multi-output (MIMO) multi-user downlink system is investigated. To achieve efficient beam tracking in near field, the trajectories of mobile users (MUs) are first predicted, and then successive hybrid beamfocusing with data stream allocation is performed to maximize the throughput. Specifically, a digit-aware location prediction framework based on Transformer is proposed to predict the subsequent locations of MUs. Through attending to the individual decimal digits of the MUs' locations, the prediction error can be reduced from the digit perspective. We then propose a dual-tiered proximal policy optimization algorithm to learn the adaptive hybrid beam-focusing and data stream allocation according to the predicted MUs' movement. The policy of agent is hierarchically designed for effective dimensionality reduction of the large-scale action space. The numerical results demonstrate that 1) Our proposed algorithms outperform the baselines in terms of throughput with high predictive accuracy and beamfocusing gain; 2) The proposed beam tracking scheme can achieve a similar throughput to the perfect CSI scheme, while the performance gap of the non-tracking scheme is 53.2%; 3) Compared to the fixed data stream allocation, the proposed adaptive data stream allocation benefits a performance gain which escalates with an increasing number of total data streams.

Active travel is an essential component in intelligent transportation systems. Cycling, as a form of active travel, shares the road space with motorised traffic which often affects the cyclists' safety and comfort and therefore peoples' propensity to uptake cycling instead of driving. This paper presents a unique dataset, collected by cyclists across London, that includes video footage, accelerometer, GPS, and gyroscope data. The dataset is then labelled by an independent group of London cyclists to rank the safety level of each frame and to identify objects in the cyclist's field of vision that might affect their experience. Furthermore, in this dataset, the quality of the road is measured by the international roughness index of the surface, which indicates the comfort of cycling on the road. The dataset11https://github.com/Chiayen0503/AllTheDocks_Dataset/tree/main will be made available for open access in the hope of motivating more research in this area to underpin the requirements for cyclists' safety and comfort and encourage more people to replace vehicle travel with cycling.

A novel non-orthogonal semantic communication (NSC) framework is proposed for facilitating high-efficiency multi-user semantic communications. The NSC technique enables non-orthogonal semantic streams among users by sharing the same resource block. A semantic superposition coding (SSC) and a semantic interference tolerated (SIT) decoding paradigm are proposed for the NSC transmitters and receivers, respectively. SSC encoders are a type of joint source-channel encoder enabled by deep learning (DL), which aims to superpose the semantic information for different users to a piece of semantic feature sequence. An SSC encoder at the access point (AP) is paired with several SIT decoders at different user equipment. By jointly training the SSC encoder and all SIT decoders, the SIT decoders can identify the desired semantic information for each user, and the semantic interferences introduced by SSC are mitigated. Simulation results reveal that the proposed NSC scheme considerably improves transmission efficiency. Meanwhile, at high compression ratios, the NSC scheme outperforms conventional orthogonal semantic communications in terms of accuracy gains.

A multiple unmanned aerial vehicle (UAV) enabled integrated sensing and communication (ISAC) system is investigated. In contrast to existing UAV-enabled ISAC systems assuming static users or 2-D UAV trajectory, we consider a practical roaming user scenario and a 3-D deployment for UAVs. Then, a joint trajectory and beamforming optimization problem is formulated for maximizing the long-term sum data rate, subject to the transmitting power constraint and ensuring beam pattern gain constraint for sensing target. To address the challenge caused by the dynamic and high dimensionality features, multiagent reinforcement learning (MARL) is employed for this partial observation Markov decision process (POMDP) problem. We proposed a two-step approach for against the dynamic scenario: 1) a K-means-based hierarchical user association algorithm is proposed to renew the user association periodically and 2) a hybrid reward multiagent proximal policy optimization (HR-MAPPO) algorithm is proposed, which decomposes the complex combined reward into a team reward and an individual reward. HR-MAPPO introduces a hyperparameter to control the proportion of team/individual action. Numerical results demonstrate that the proposed HR-MAPPO algorithm can outperform the conventional single-agent and multiagent RL algorithms by maintaining high scores on both the sum data rate and beam pattern gain.

A simultaneously transmitting and reflecting surface (STARS) assisted multi-user downlink multiple-input signal-output (MISO) multi-cellular edge caching system is investigated. The deployment of STARS enhances the coverage of base stations (BSs), particularly at cellular boundaries. However, this advancement introduces a complex user association issue that necessitates the consideration of both caching state and channel state information (CSI). In this paper, we formulate a joint optimization problem involving content caching, user association, active beamforming at BS, and passive beamforming at STARS for minimizing long-term power consumption. We propose two algorithms for the formulated problem: 1) A two time-scale cooperative twin delayed deep deterministic policy gradients (TD3). Considering the distinct time scales of the pushing and delivering phases in edge caching, the Markov decision process (MDP) models of dual time scales are constructed and two deep reinforcement learning (DRL) agents work together to jointly address the optimization problem. 2) A bio-inspired DRL framework, especially, a particle swarm optimization (PSO)-inspired TD3 algorithm is introduced in detail. Inspired by the behavior of the biological population in nature, this algorithm regards agents as individuals and enables the concurrent training of multiple agents while they interact with global information via a biological population information interaction mode, thereby enhancing the performance of power optimization. The numerical results demonstrate that the STARS-assisted multi-cellular edge caching system has advantages over traditional cellular systems, especially in scenarios where the number of mobile users and Zipf skewness factor is large. Moreover, the proposed two time-scale cooperative TD3 and PSO-inspired TD3 algorithms are superior in reducing network power consumption than conventional TD3.

A simultaneously transmitting and reflecting surface (STARS) enabled edge caching system is proposed for reducing backhaul traffic and ensuring the quality of service. A novel Caching-at-STARS structure, where a dedicated smart controller and cache memory are installed at the STARS, is proposed to satisfy user demands with fewer hops and desired channel conditions. Then, a joint caching replacement and information-centric hybrid beamforming optimization problem is formulated for minimizing the network power consumption. As long-term decision processes, the optimization problems based on independent and coupled phase-shift models of Caching-at-STARS contain both continuous and discrete decision variables, and are suitable for solving with deep reinforcement learning (DRL) algorithm. For the independent phase-shift Caching-at-STARS model, we develop a frequency-aware based twin delayed deep deterministic policy gradient (FA-TD3) algorithm that leverages user historical request information to serialize high-dimensional caching replacement decision variables. For the coupled phase-shift Caching-at-STARS model, we conceive a cooperative TD3 & deep-Q network (TD3-DQN) algorithm comprised of FA-TD3 and DQN agents to decide on continuous and discrete variables respectively by observing the network external and internal environment. The numerical results demonstrate that: 1) The Caching-at-STARS-enabled edge caching system has advantages over traditional edge caching, especially in scenarios where Zipf skewness factors or cache capacity is large; 2) Caching-at-STARS outperforms the RIS-assisted edge caching systems; 3) The proposed FA-TD3 and cooperative TD3-DQN algorithms are superior in reducing network power consumption than conventional TD3.

In this paper, we propose a novel interference-suppressed semantic-aware non-orthogonal multiple access (IS-SNOMA) framework for downlink image transmission in the semantic-aware communication networks. The proposed IS-SNOMA is able to mitigate the inter-user interference in the non-orthogonal transmission for multiple semantic-oriented users (SU) or the coexistence of SUs and bit-oriented users (BU). 1) For the homogeneous transmission of semantic streams, we propose a pair of novel semantic difference (SeD) aware IS-SNOMA transceivers to accommodate multiple SUs over the same resource block. A novel SeD-aware superposition coding (SeDSC) technique and SeD-aware successive interference cancellation (SeDSIC) technique are specially designed to mitigate the semantic-level interference. 2) For the heterogeneous transmission of semantic-bit streams, we develop a pair of syntactic difference (SyD) aware IS-SNOMA transceivers to multiplex channels for SUs and BUs. The heterogeneous semantic symbols and bit sequences are superposed and separated with the proposed SyD-aware SC technique (SyDSC) and SyD-aware SIC technique (SyDSIC), respectively. Simulation results demonstrate the advantages of the proposed frameworks in improving the communication efficiency of both SUs and BUs, compared with OMA and NOMA-aided transmission benchmarks.

A multi-user heterogeneous semantic communication (SemCom) and bit communication (BitCom) system is investigated. Each user can be served via either SemCom or BitCom for demanding semantic or bit data. Orthogonal/non-orthogonal multiple access (OMA/NOMA) techniques are employed to provide access for multiple users. Channel-based and user demand-based transmission protocols are proposed, where a joint optimization problem of the communication mode selection, frequency bandwidth and power allocation, and NOMA user pairing is formulated to maximize the long-term (equivalent) semantic throughput and user satisfaction, respectively. To solve the formulated problems: 1) For channel-based transmission, a twin-delayed deep deterministic policy gradient with reference neuron enhanced Softmax (TD3-RNS) algorithm is proposed, where a fixed-value neuron is invoked to improve the training efficiency; 2) For user demand-based transmission, a transfer TD3-RNS (T2D3-RNS) algorithm is proposed, where the learned policy is transferred to address the sparse rewards and perverse incentive problem caused by the optimization objective and reward shaping, respectively. Simulation results demonstrate that: i) The proposed heterogeneous scheme outperforms the baselines which merely use SemCom or BitCom; ii) Compared to OMA, NOMA is more compatible with the proposed heterogeneous scheme; and iii) The proposed algorithms outperform the benchmarks in channel-based and user demand-based transmission, respectively.

A novel Caching-at-STARS structure, where dedicated smart controller and cache memory are installed at the STARS, is proposed to satisfy user demands with fewer hops and desired channel condition. Then, a joint caching replacement and information-centric hybrid beamforming optimization problem is formulated for minimizing the network power consumption. We conceive a cooperative twin delayed deep deterministic policy gradient & deep-Q network (TD3-DQN) algorithm comprised by TD3 and DQN agents to decide on continuous and discrete variables respectively by observing the network external and internal environment. The numerical results demonstrate that: 1) The Caching-at-STARS-enabled edge caching system has advantages over traditional edge caching, especially in scenarios where Zipf skewness factors or cache capacity is large; 2) STARS outperforms RIS significantly in edge caching systems; 3) The proposed cooperative TD3-DQN algorithms is superior in reducing network power consumption than conventional TD3.

A large reconfigurable intelligent surface (LRIS) assisted wireless communication system is investigated in this paper. The increased aperture size and reconfigurable element number of LRIS bring new challenges, including limited incident beam coverage on LRIS, near-field signal propagation, and high beamforming complexity. Against these challenges, a two-step low-complexity beamforming approach is proposed, where a deep reinforcement learning (DRL) algorithm is invoked for determining the optimal beam direction, and a codebook based on the geometric channel state information is designed to map the direction to the beamforming matrixes. The proposed approach not only reduces the computational complexity, but also exploits the geometric channel of BS-LRIS to reduce the channel estimation complexity caused by LRIS. Simulation results indicate that the LRIS can further reduce power consumption compared to the small-size RIS. Meanwhile, the proposed joint codebook-DRL approach achieves a counterbalance compared to the sheer DRL algorithm with lower complexity.

As an advanced version of the Internet of Things (IoT), the Internet of Everything (IoE) network enables massive access for users and machine-type devices. A surge in the number of connected devices results in a stringent requirement for multiple access. To meet this challenge, a beamforming scheme in a downlink multiple-input and single-output (MISO) transmission is investigated, where an overloaded base station (BS) is committed to serving every user. Due to the deficiency of the number of antennas, the overload user equipment (OUE) needs to ride on an existing beam to connect to the BS. The optimization goal in this work is to maximize the energy efficiency (EE) of the system by joint optimizing for the selection of the shared beam and power allocation, subject to a constraint of the total power allocated to each beam. To tackle the formulated problem, a proximal policy optimization (PPO) based solution is proposed for jointly determining the beam choice and power allocation. Simulation results revealed that an average EE gain of 27.75 % can be achieved by the proposed solution compared with a spectral efficiency (SE) based method. Furthermore, the simulation with an increasing number of overload users highlights the performance of the joint optimization.

A novel framework of reconfigurable intelligent surfaces (RISs) enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable to provide NOMA with artificially controlled channels, which can be a beneficial condition for NOMA networks. To optimize the sum rate of all users, a federated learning enhanced deep deterministic policy gradient (FL-DDPG) algorithm is proposed to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. Our simulation results indicate that the mobile RIS scheme can provide about three times data rate gain compare to the fixed RIS. Moreover, the NOMA scheme is capable to achieve a significant data rate gain in contrast with the OMA scheme. Finally, the FL-DDPG algorithm has a superior convergence rate and optimization performance than that of the independent training framework.

A reconfigurable intelligent surface (RIS)-assisted multi-user downlink communication system over fading channels is investigated, where both non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) schemes are employed. In particular, the time overhead for configuring the RIS reflective elements at the beginning of each fading channel is considered. The optimization goal is maximizing the effective throughput of the entire transmission period by jointly optimizing the phase shift of the RIS and the power allocation of the AP for each channel block. In an effort to solve the formulated problem and fill the research vacancy of the performance comparison between different machine learning tools in wireless networks, a deep learning (DL) approach and a reinforcement learning (RL) approach are proposed and their representative superiority and inferiority are investigated. The DL approach can locate the optimal phase shifts with the deep neural network fitting as well as the corresponding power allocation for each user. From the perspective of long-term reward, the phase shift control with configuration overhead can be regarded as a Markov decision process and the RL algorithm is proficient in solving such problems with the assistance of the Bellman equation. The numerical results indicate that: 1) From the perspective of the wireless network, NOMA can achieve a throughput gain of about 42% compared with OMA; 2) The well-trained RL and DL agents are able to achieve the same performance in Rician channel, while RL is superior in the Rayleigh channel; 3) The DL approach has lower complexity and faster convergence, while the RL approach has preferable strategy flexibility.

A simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted multi-user downlink multiple-input single-output (MISO) communication system is investigated. In contrast to the existing ideal STAR-RIS model assuming an independent transmission and reflection phase-shift control, a practical coupled phase-shift model is considered. Then, a joint active and passive beamforming optimization problem is formulated for minimizing the long-term transmission power consumption, subject to the coupled phase-shift constraint and the minimum data rate constraint. Despite the coupled nature of the phase-shift model, the formulated problem is solved by invoking a hybrid continuous and discrete phase-shift control policy. Inspired by this observation, a pair of hybrid reinforcement learning (RL) algorithms, namely the hybrid deep deterministic policy gradient (hybrid DDPG) algorithm and the joint DDPG & deep-Q network (DDPG-DQN) based algorithm are proposed. The hybrid DDPG algorithm controls the associated high-dimensional continuous and discrete actions by relying on the hybrid action mapping. By contrast, the joint DDPG-DQN algorithm constructs two Markov decision processes (MDPs) relying on an inner and an outer environment, thereby amalgamating the two agents to accomplish a joint hybrid control. Simulation results demonstrate that the STAR-RIS has superiority over other conventional RISs in terms of its energy consumption. Furthermore, both the proposed algorithms outperform the baseline DDPG algorithm, and the joint DDPG-DQN algorithm achieves a superior performance, albeit at an increased computational complexity.

A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs), while non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. The optimization problem of joint three-dimensional (3D) trajectory design and power allocation is formulated for maximizing the throughput. Since ground mobile users are considered as roaming continuously, the UAVs need to be re-deployed timely based on the movement of users. In an effort to solve this pertinent dynamic problem, a K-means based clustering algorithm is first adopted for periodically partitioning users. Afterward, a mutual deep Q-network (MDQN) algorithm is proposed to jointly determine the optimal 3D trajectory and power allocation of UAVs. In contrast to the conventional deep Q-network (DQN) algorithm, the MDQN algorithm enables the experience of multi-agent to be input into a shared neural network to shorten the training time with the assistance of state abstraction. Numerical results demonstrate that: 1) the proposed MDQN algorithm is capable of converging under minor constraints and has a faster convergence rate than the conventional DQN algorithm in the multi-agent case; 2) The achievable sum rate of the NOMA enhanced UAV network is 23% superior to the case of orthogonal multiple access (OMA); 3) By designing the optimal 3D trajectory of UAVs with the MDON algorithm, the sum rate of the network enjoys 142% and 56% gains than invoking the circular trajectory and the 2D trajectory, respectively.

A novel framework of reconfigurable intelligent surfaces (RISs)-enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enable mobility of the RIS and enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable of providing NOMA with artificial controlled channel conditions, which can be seen as a beneficial operation condition to obtain NOMA gains. To optimize the sum rate of all users, a deep deterministic policy gradient (DDPG) algorithm is invoked to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. In order to improve the efficiency and effectiveness of agent training for the DDPG agents, a federated learning (FL) concept is adopted to enable multiple agents to simultaneously explore similar environments and exchange experiences. We also proved that with the same random exploring policy, the FL armed deep reinforcement learning (DRL) agents can theoretically obtain a reward gain comparing to the independent agents. Our simulation results indicate that the mobile RIS scheme can significantly outperform the fixed RIS paradigm, which provides about three times data rate gain compared to the fixed RIS paradigm. Moreover, the NOMA scheme is capable of achieving a gain of 42% in contrast with the OMA scheme in terms of the sum rate. Finally, the multi-cell simulation proved that the FL enhanced DDPG algorithm has a superior convergence rate and optimization performance than the independent training framework.

A communication enabled indoor intelligent robots (IRs) service framework is proposed, where non-orthogonal multiple access (NOMA) technique is adopted to enable highly reliable communications. In cooperation with the ultramodern indoor channel model recently proposed by the International Telecommunication Union (ITU), the Lego modeling method is proposed, which can deterministically describe the indoor layout and channel state in order to construct the radio map. The investigated radio map is invoked as a virtual environment to train the reinforcement learning agent, which can save training time and hardware costs. Build on the proposed communication model, motions of IRs who need to reach designated mission destinations and their corresponding down-link power allocation policy are jointly optimized to maximize the mission efficiency and communication reliability of IRs. In an effort to solve this optimization problem, a novel reinforcement learning approach named deep transfer deterministic policy gradient (DT-DPG) algorithm is proposed. Our simulation results demonstrate in the following: 1) with the aid of NOMA techniques, the communication reliability of IRs is effectively improved; 2) radio map is qualified to be a virtual training environment, and its statistical channel state information improves training efficiency by about 30%; 3) proposed DT-DPG algorithm is superior to the conventional deep deterministic policy gradient (DDPG) algorithm in terms of optimization performance, training time, and anti-local optimum ability.