Multimodal Large Language Models (MLLMs) enhance collaboration in Extended Reality (XR) environments by enabling flexible object and animation creation through the combination of natural language and visual inputs. However, visual data captured by XR headsets includes real-world backgrounds that may contain irrelevant or sensitive user information, such as credit cards left on the table or facial identities of other users. Uploading those frames to cloud-based MLLMs poses serious privacy risks, particularly when such data is processed without explicit user consent. Additionally, existing colocation and synchronization mechanisms in commercial XR APIs rely on time-consuming, privacy-invasive environment scanning and struggle to adapt to the highly dynamic nature of MLLM-integrated XR environments. In this paper, we propose PRISM-XR, a novel framework that facilitates multi-user collaboration in XR by providing privacy-aware MLLM integration. PRISM-XR employs intelligent frame preprocessing on the edge server to filter sensitive data and remove irrelevant context before communicating with cloud generative AI models. Additionally, we introduce a lightweight registration process and a fully customizable content-sharing mechanism to enable efficient, accurate, and privacy-preserving content synchronization among users. Our numerical evaluation results indicate that the proposed platform achieves nearly 90% accuracy in fulfilling user requests and less than 0.27 seconds registration time while maintaining spatial inconsistencies of less than 3.5 cm. Furthermore, we conducted an IRB-approved user study with 28 participants, demonstrating that our system could automatically filter highly sensitive objects in over 90% of scenarios while maintaining strong overall usability.

In this work, we investigate a movable antenna (MA) enhanced secure uplink non-orthogonal multiple access (NOMA) network, where multiple legitimate users, each equipped with an MA, transmit confidential information to the base station in the presence of an eavesdropper. Our goal is to maximize the system's secrecy sum rate by jointly optimizing the positions of MAs, the transmit power of each user, and the receive combining vectors at the base station. To tackle the formulated non-convex problem, we first transform it into a more tractable form and then develop a customized heuristics algorithm by invoking particle swarm optimization (PSO) to obtain a high-quality suboptimal solution. Simulation results validate the efficacy of our proposed algorithm, demonstrating its superior performance over conventional fixed-position antenna (FPA) systems and other baselines.

Recent advances in reconfigurable antennas have led to the new concept of the fluid antenna system (FAS) for shape and position flexibility, as another degree of freedom for wireless communication enhancement. This paper explores the integration of a transmit FAS array for hybrid beamforming (HBF) into a reconfigurable intelligent surface (RIS)-assisted communication architecture for multiuser communications in the downlink, corresponding to the downlink RIS-assisted multiuser multiple-input single-output (MISO) FAS model (Tx RIS-assisted-MISO-FAS). By considering Rician channel fading, we formulate a sum-rate maximization optimization problem to alternately optimize the HBF matrix, the RIS phase-shift matrix, and the FAS position. Due to the strong coupling of multiple optimization variables, the multi-fractional summation in the sum-rate expression, the modulus-1 limitation of analog phase shifters and RIS, and the antenna position variables appearing in the exponent, this problem is highly non-convex, which is addressed through the block coordinate descent (BCD) framework in conjunction with semidefinite relaxation (SDR) and majorization-minimization (MM) methods. To reduce the computational complexity, we then propose a low-complexity grating-lobe (GL)-based telescopic-FA (TFA) system with multiple delicately deployed RISs under the sub-connected HBF architecture and the line-of-sight (LoS)-dominant channel condition, to allow closed-form solutions for the HBF and TFA position. Our simulation results illustrate that the former optimization scheme significantly enhances the achievable rate of the proposed system, while the GL-based TFA scheme also provides a considerable gain over conventional fixed-position antenna (FPA) systems, requiring statistical channel state information (CSI) only and with low computational complexity.

Stacked intelligent metasurfaces (SIMs), which integrate multiple programmable metasurface layers, have recently emerged as a promising technology for advanced wave-domain signal processing. SIMs benefit from flexible spatial degree-of-freedom (DoF) while reducing the requirement for costly radio-frequency (RF) chains. However, current state-of-the-art SIM designs face challenges such as complex phase shift optimization and energy attenuation from multiple layers. To address these aspects, we propose incorporating meta-fibers into SIMs, with the aim of reducing the number of layers and enhancing the energy efficiency. First, we introduce a meta-fiber-connected 2-layer SIM that exhibits the same flexible signal processing capabilities as conventional multi-layer structures, and explains the operating principle. Subsequently, we formulate and solve the optimization problem of minimizing the mean square error (MSE) between the SIM channel and the desired channel matrices. Specifically, by designing the phase shifts of the meta-atoms associated with the transmitting-SIM and receiving-SIM, a non-interference system with parallel subchannels is established. In order to reduce the computational complexity, a closed-form expression for each phase shift at each iteration of an alternating optimization (AO) algorithm is proposed. We show that the proposed algorithm is applicable to conventional multi-layer SIMs. The channel capacity bound and computational complexity are analyzed to provide design insights. Finally, numerical results are illustrated, demonstrating that the proposed two-layer SIM with meta-fiber achieves over a 25% improvement in channel capacity while reducing the total number of meta-atoms by 59% as compared with a conventional seven-layer SIM.

Artificial noise (AN) has been recognized as an effective physical-layer security scheme impairing the eavesdropper (Eve). Recently, artificial noise elimination (ANE) has emerged as a promising strategy to mitigate the impact of AN at Eves. However, conventional ANE schemes rely on prior knowledge, such as legitimate channel state information (CSI) or classification information, which may limit their practical applicability. To address these practical challenges, we propose an ANE scheme beyond prior knowledge (BPK) by leveraging machine learning algorithms. Firstly, a coarse projection is applied to partially eliminate the impact of AN using maximum likelihood estimation on the equivalent AN matrix. Secondly, a density clustering algorithm is introduced to obtain classification information based on the coarsely-projected observed vectors. Thirdly, a generalized principal component analysis (PCA)-based ANE algorithm is developed to effectively mitigate the residual AN using the obtained classification information. Furthermore, the artificial-noise-to-signal ratio (ANSR) and computational complexity are analyzed for performance revaluation, and a redefinition of several AN design principles is provided for scenarios involving a powerful Eve equipped with the BPK-ANE scheme by deriving the validity boundary. Finally, numerical results reveal key insights into four principles of AN: 1) Allocating less power to AN; 2) Reducing the randomness of AN; 3) Increasing the number of transmit antennas; and 4) Increasing the modulation order.

Mixed Reality (MR)-aided operation overlays digital objects on the physical world to provide a more immersive and intuitive operation process. A primary challenge is the precise and fast auto-verification of whether the user follows MR guidance by comparing frames before and after each operation. The pre-operation frame includes virtual guiding objects, while the post-operation frame contains physical counterparts. Existing approaches fall short of accounting for the discrepancies between physical and virtual objects due to imperfect 3D modeling or lighting estimation. In this paper, we propose EVER: an edge-assisted auto-verification system for mobile MR-aided operations. Unlike traditional frame-based similarity comparisons, EVER leverages the segmentation model and rendering pipeline adapted to the unique attributes of frames with physical pieces and those with their virtual counterparts; it adopts a threshold-based strategy using Intersection over Union (IoU) metrics for accurate auto-verification. To ensure fast auto-verification and low energy consumption, EVER offloads compute-intensive tasks to an edge server. Through comprehensive evaluations of public datasets and custom datasets with practical implementation, EVER achieves over 90% verification accuracy within 100 milliseconds (significantly faster than average human reaction time of approximately 273 milliseconds), while consuming only minimal additional computational resources and energy compared to a system without auto-verification.

Inference using deep neural networks on mobile devices has been an active area of research in recent years. The design of a deep learning inference framework targeted for mobile devices needs to consider various factors, such as the limited computational capacity of the devices, low power budget, varied memory access methods, and I/O bus bandwidth governed by the underlying processor's architecture. Furthermore, integrating an inference framework with time-sensitive applications - such as games and video-based software to perform tasks like ray tracing denoising and video processing - introduces the need to minimize data movement between processors and increase data locality in the target processor. In this paper, we propose Shader Neural Network (ShaderNN), an OpenGL-based, fast, and power-efficient inference framework designed for mobile devices to address these challenges. Our contributions include the following: (1) the texture-based input/output provides an efficient, zero-copy integration with real-time graphics pipelines or image processing applications, thereby saving expensive data transfers between CPU and GPU, which are unavoidable in most existing inference engines; (2) we are the first to leverage fragment shaders based on the OpenGL backend in neural network inference operators, which has an advantage in deploying parametrically small neural network models; (3) a hybrid implementation of the compute shader and fragment shader is proposed that enables layer-level shader selection to boost performance; and (4) we utilize OpenGL features - such as normalization, interpolation and texture padding - to improve performance. Experiments illustrate the favorable performance of ShaderNN over other popular on-device deep learning frameworks such as TensorFlow-Lite on the latest mobile devices powered by Qualcomm and MediaTek chips. A case study further demonstrates the usability and integration of the ShaderNN framework with a media processing Android application seamlessly. ShaderNN is available open source at Github (https://github.com/inferenceengine/shadernn).

Directional modulation (DM) is a physical layer security (PLS) technique implemented at the transmitter, leveraging antenna arrays to ensure secure communications. Through a process of spatial precoding between transceivers to transmit signals in specific directions, DM is capable of disrupting communications in unintended directions to prevent eavesdropping. In general, recent progress in the development of multiple-input multiple-output (MIMO) systems, including advanced radio frequency (RF), antenna technologies, along with innovative precoding algorithms, has enhanced the capabilities of DM techniques, leading to a multitude of robust DM variants. Hence, this survey aims to offer a comprehensive overview of DM, covering its fundamentals, promising variants, applications, hardware implementations, and future trends. Initially, the basic principle of DM is outlined in a general manner for subsequent comprehension. Subsequently, the large family of DM techniques is categorized into distinct variants based on the types of transmitting arrays. Next, we give a comprehensive survey of DM in common wireless scenarios, including multi-user (MU), relay, Internet of Things (IoT), and non-orthogonal access (NOMA) networks. Furthermore, we provide an illustration of DM system implementations, encompassing foundational architectures and cost-effective hardware realizations. Finally, concerning the unresolved challenges and current research focal points in DM, we present future research directions that merit further exploration and reference.

Directional modulation (DM), as a multi-antenna-based physical layer security (PLS) technology toward millimeter wave (mmWave) scenarios, is capable of guaranteeing effective and secure transmissions by simultaneously performing beamforming and artificial noise (AN). Over the past decade, the DM architecture has changed from a fully analog (FA) one to a fully digital (FD) one. Additionally, the hybrid architecture, which effectively balances both performance and hardware considerations, has unfortunately received limited attention regarding the integration of DM within its framework. In this paper, we intend to give a detailed study of DM technology under the still imperfect FA and more effective hybrid architectures. Specifically, for a single-user (SU) multiple-input single-output (MISO) FA-DM systems, considering the constant modulus constraint (CMC) of phase shifters (PSs) as well as the power allocation issue, we propose a low-complexity DM precoder with closed-form solutions by maximizing the entropy of precoder (EoP). On the other hand, optimization problems are cast and discussed under different RF chain number configurations for SU multiple-input multiple-output (MIMO) hybrid-DM systems. Specifically, as the RF chain count decreases from twice the data stream number to match it, we all use optimization to achieve the corresponding optimal or near-optimal solutions. Finally, simulation results verify our aforementioned analysis and demonstrate the superiority of the proposed FA and hybrid-DM algorithms in terms of secrecy rate and bit error rate (BER) compared to the counterparts when PSs with low resolution are used.

The integration of Large Language Models (LLMs) like GPT-4 with Extended Reality (XR) technologies offers the potential to build truly immersive XR environments that interact with human users through natural language, e.g., generating and animating 3D scenes from audio inputs. However, the complexity of XR environments makes it difficult to accurately extract relevant contextual data and scene/object parameters from an overwhelming volume of XR artifacts. It leads to not only increased costs with pay-per-use models, but also elevated levels of generation errors. Moreover, existing approaches focusing on coding script generation are often prone to generation errors, resulting in flawed or invalid scripts, application crashes, and ultimately a degraded user experience. To overcome these challenges, we introduce LLMER, a novel framework that creates interactive XR worlds using JSON data generated by LLMs. Unlike prior approaches focusing on coding script generation, LLMER translates natural language inputs into JSON data, significantly reducing the likelihood of application crashes and processing latency. It employs a multi-stage strategy to supply only the essential contextual information adapted to the user's request and features multiple modules designed for various XR tasks. Our preliminary user study reveals the effectiveness of the proposed system, with over 80% reduction in consumed tokens and around 60% reduction in task completion time compared to state-of-the-art approaches. The analysis of users' feedback also illuminates a series of directions for further optimization.

The Internet of Things (IoT) interconnects a vast number of physical devices with each other and enables remote monitoring and control of the physical system. This, together with the recent progress of virtual reality (VR) technology, provides an immersive experience for the user to perform remote monitoring and control as if she observes and controls the physical system in person. In this demo, we develop an immersive remote monitoring and control system consisting of 3D virtual monitoring and control panel and the physical water pump system. Users can remotely control and monitor the water level of the physical water pump system through the 3D virtual panel in real-time. We use Fourier transform to filter out the noise when the water level remains static and the exponential running average method to make the collected data more smooth when the water level dynamically changes. The experimental evaluations demonstrate that the difference between the physical and virtual water levels is small in both static and dynamic water levels.

In this contribution, the concepts of polarization modulation (PM) and spatial modulation (SM) are integrated, to reap their respective advantages toward single-radio frequency (RF) multiple-input multiple-output (MIMO) transmissions. In the so-called polarization and spatial modulation (PSM) system, the information is conveyed by activated antenna indices as well as polarization modulated symbols, while at the receiver, a low-complexity near-optimal detection algorithm based on compressive sensing (CS) is proposed. Furthermore, a closed-form union bound of the average bit-error rate (BER) over fading channels is quantified by theoretical derivation, which is then extended to the case of spatial correlation (SC) as well as channel estimation error (CSE) toward practical use. Finally, our simulation results demonstrate the superiority of the developed PSM system over conventional PM in terms of BER performance over various fading channels.

Virtual Reality (VR), together with the network infrastructure, can provide an interactive and immersive experience for multiple users simultaneously and thus enables collaborative VR applications (e.g., VR-based classroom). However, the satisfactory user experience requires not only high-resolution panoramic image rendering but also extremely low latency and seamless user experience. Besides, the competition for limited network resources (e.g., multiple users share the total limited bandwidth) poses a significant challenge to collaborative user experience, in particular under the wireless network with time-varying capacities. While existing works have tackled some of these challenges, a principled design considering all those factors is still missing. In this paper, we formulate a combinatorial optimization problem to maximize the Quality of Experience (QoE), defined as the linear combination of the quality, the average VR content delivery delay, and variance of the quality over a finite time horizon. In particular, we incorporate the influence of imperfect motion prediction when considering the quality of the perceived contents. However, the optimal solution to this problem can not be implemented in real-time since it relies on future decisions. Then, we decompose the optimization problem into a series of combinatorial optimization in each time slot and develop a low-complexity algorithm that can achieve at least 1/2 of the optimal value. Despite this, the trace-based simulation results reveal that our algorithm performs very close to the decomposed optimal offline solution. Furthermore, we implement our proposed algorithm in a practical system with commercial mobile devices and demonstrate its superior performance over state-of-the-art algorithms. We open-source our implementations on https://github.com/SNeC-Lab-PSU/ICDCS-CollaborativeVR.

Interactive panoramic scene delivery not only consumes 4∼6× more bandwidth than traditional video streaming of the same resolution but also requires timely displaying the delivered content to ensure smooth interaction. Since users can only see roughly 20% of the entire scene at a time (called the viewport), it is sufficient to deliver the relevant portion of the panoramic scene if we can accurately predict the user’s motion. It is customary to deliver a portion larger than the viewport to tolerate inaccurate predictions. Intuitively, the larger the delivered portion, the higher the prediction accuracy and lower the wireless transmission success probability. The goal is to select an appropriate delivery portion to maximize system throughput. We formulate this problem as a multi-armed bandit problem and use the classical Kullback-Leibler Upper Confidence Bound (KL-UCB) algorithm for the portion selection. We further develop a novel variant of the KL-UCB algorithm that effectively leverages two-level feedback (i.e., both prediction and transmission outcomes) after each decision on the selected portion and show its asymptotical optimality, which may be of independent interest by itself. We demonstrate the superior performance of our proposed algorithms over existing heuristic methods using both synthetic simulations and real experimental evaluations.

Abstract Recently, the physical limitation of range‐domain security guarantee for directional modulation with frequency diverse array was disclosed in Ding et al. ( IEEE Access 8, 63302–63309 (2020)). Therefore, to recreate this significant secrecy realisation in both direction and range domain, the authors conceive an artificial noise aided directional modulation scheme via reconfigurable intelligent surface. Specifically, the angle of departure from the reconfigurable intelligent surface to co‐direction receivers varies with distance, which provides the freedom to distinguish users in the range domain. With locations of the active eavesdroppers, an optimisation problem is further formulated to maximise the secrecy rate, which is then solved by applying genetic algorithm and alternating optimisation. On the other hand, when the eavesdroppers are silent, we employ maximum‐ratio transmission precoding and artificial noise to guarantee information security. Finally, simulation results demonstrate that, due to the aid of reconfigurable intelligent surface, the developed directional modulation structure can robustly guarantee the information security in the range domain, whether the information of eavesdroppers is achieved.

Multi-user panoramic video streaming demands 4~6× bandwidth of a regular video with the same resolution, which poses a significant challenge on the wireless scheduling design to achieve desired performance. On the other hand, recent studies reveal that one can effectively predict the user's Field-of-View (FoV) and thus simply deliver the corresponding portion instead of the entire scenes. Motivated by this important fact, we aim to employ autoregressive process for motion prediction and analytically characterize the user's successful viewing probability as a function of the delivered portion. Then, we consider the problem of wireless scheduling design with the goal of maximizing application-level throughput (i.e., average rate for successfully viewing the desired content) and service regularity performance (i.e., how often each user gets successful views) subject to the minimum required service rate and wireless interference constraints. As such, we incorporate users' successful viewing probabilities into our scheduling design and develop a scheduling algorithm that not only asymptotically achieves the optimal application-level throughput but also provides service regularity guarantees. Finally, we perform simulations to demonstrate the efficiency of our proposed algorithm using a real dataset of users' head motion.