Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Tao Wang, Linfeng Zhang, Biqing Qi, Bowen Zhou. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Yuhua Jiang, Shuang Cheng, Yihao Liu, Ermo Hua, Che Jiang, Weigao Sun, Yu Cheng, Feifei Gao, Biqing Qi, Bowen Zhou. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Bo Lin, Huanming Zhang, Yuhua Jiang, Yucong Wang, Tengyu Zhang, Shaoqiang Yan, Hongyao Li, Yihong Liu, Feifei Gao. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Exploration in sparse reward environments remains a significant challenge in reinforcement learning, particularly in Contextual Markov Decision Processes (CMDPs), where environments differ across episodes. Existing episodic intrinsic motivation methods for CMDPs primarily rely on count-based approaches, which are ineffective in large state spaces, or on similarity-based methods that lack appropriate metrics for state comparison. To address these shortcomings, we propose Episodic Novelty Through Temporal Distance (ETD), a novel approach that introduces temporal distance as a robust metric for state similarity and intrinsic reward computation. By employing contrastive learning, ETD accurately estimates temporal distances and derives intrinsic rewards based on the novelty of states within the current episode. Extensive experiments on various benchmark tasks demonstrate that ETD significantly outperforms state-of-the-art methods, highlighting its effectiveness in enhancing exploration in sparse reward CMDPs.

Research in offline reinforcement learning (RL) marks a paradigm shift in RL. However, a critical yet under-investigated aspect of offline RL is determining the subset of the offline dataset, which is used to improve algorithm performance while accelerating algorithm training. Moreover, the size of reduced datasets can uncover the requisite offline data volume essential for addressing analogous challenges. Based on the above considerations, we propose identifying Reduced Datasets for Offline RL (ReDOR) by formulating it as a gradient approximation optimization problem. We prove that the common actor-critic framework in reinforcement learning can be transformed into a submodular objective. This insight enables us to construct a subset by adopting the orthogonal matching pursuit (OMP). Specifically, we have made several critical modifications to OMP to enable successful adaptation with Offline RL algorithms. The experimental results indicate that the data subsets constructed by the ReDOR can significantly improve algorithm performance with low computational complexity.

Integrated sensing and communications (ISAC) has emerged as a transformative paradigm for next-generation wireless systems. In this paper, we present a novel ISAC scheme that leverages the diffusion Schr¨odinger bridge (DSB) to realize the sensing of electromagnetic (EM) property of a target as well as the reconstruction of the wireless channel. The DSB framework connects EM property sensing and channel reconstruction by establishing a bidirectional process: the forward process transforms the distribution of EM property into the channel distribution, while the reverse process reconstructs the EM property from the channel. To handle the difference in dimensionality between the high-dimensional sensing channel and the lower-dimensional EM property, we generate latent representations using an autoencoder network. The autoencoder compresses the sensing channel into a latent space that retains essential features, which incorporates positional embeddings to process spatial context. The simulation results demonstrate the effectiveness of the proposed DSB framework, which achieves superior reconstruction of the targets shape, relative permittivity, and conductivity. Moreover, the proposed method can also realize accurate channel reconstruction given the EM property of the target. The dual capability of accurately sensing the EM property and reconstructing the channel across various positions within the sensing area underscores the versatility and potential of the proposed approach for broad application in future ISAC systems.

Integrated sensing and communications (ISAC) has opened up numerous game-changing opportunities for future wireless systems. In this paper, we develop a novel ISAC scheme that utilizes the diffusion model to sense the electromagnetic (EM) property of the target in a predetermined sensing area. Specifically, we first estimate the sensing channel by using both the communications and the sensing signals echoed back from the target. Then we employ the diffusion model to generate the point cloud that represents the target and thus enables 3D visualization of the target’s EM property distribution. In order to minimize the mean Chamfer distance (MCD) between the ground truth and the estimated point clouds, we further design the communications and sensing beamforming matrices under the constraint of a maximum transmit power and a minimum communications achievable rate for each user equipment (UE). Simulation results demonstrate the efficacy of the proposed method in achieving high-quality reconstruction of the target’s shape, relative permittivity, and conductivity. Besides, the proposed method can sense the EM property of the target effectively in any position of the sensing area.

Integrated sensing and communication (ISAC) has opened up numerous game-changing opportunities for future wireless systems. In this paper, we develop a novel scheme that utilizes orthogonal frequency division multiplexing (OFDM) pilot signals to sense the electromagnetic (EM) property of the target and thus identify the materials of the target. Specifically, we first establish an EM wave propagation model with Maxwell equations, where the EM property of the target is captured by a closed-form expression of the channel. We then build the mathematical model for the relative permittivity and conductivity distribution (RPCD) within a predetermined region of interest shared by multiple base stations (BSs). By leveraging the Lippmann-Schwinger equation, we propose an EM property sensing method that reconstructs the RPCD using compressive sensing techniques. This approach exploits the joint sparsity of the EM property vector, which enables the proposed method to effectively handle the high dimensionality and ill-posed nature of the inverse scattering problem. We then develop a fusion algorithm to combine data from multiple BSs, which can enhance the reconstruction accuracy of EM property by efficiently integrating diverse measurements. Moreover, the fusion is performed at the feature level of RPCD and features low transmission overhead. We further design the pilot signals that can minimize the mutual coherence of the equivalent channels and enhance the diversity of incident EM wave patterns. Simulation results demonstrate the efficacy of the proposed method in achieving high-quality RPCD reconstruction and accurate material classification.

Among the remarkable successes of Reinforcement Learning (RL), self-play algorithms have played a crucial role in solving competitive games. However, current self-play RL methods commonly optimize the agent to maximize the expected win-rates against its current or historical copies, resulting in a limited strategy style and a tendency to get stuck in local optima. To address this limitation, it is important to improve the diversity of policies, allowing the agent to break stalemates and enhance its robustness when facing with different opponents. In this paper, we present a novel perspective to promote diversity by considering that agents could have diverse risk preferences in the face of uncertainty. To achieve this, we introduce a novel reinforcement learning algorithm called Risk-sensitive Proximal Policy Optimization (RPPO), which smoothly interpolates between worst-case and best-case policy learning, enabling policy learning with desired risk preferences. Furthermore, by seamlessly integrating RPPO with population-based self-play, agents in the population optimize dynamic risk-sensitive objectives using experiences gained from playing against diverse opponents. Our empirical results demonstrate that our method achieves comparable or superior performance in competitive games and, importantly, leads to the emergence of diverse behavioral modes. Code is available at https://github.com/Jackory/RPBT.

Near field computational imaging has been recognized as a promising technique for non-destructive and highly accurate detection of the target. Meanwhile, reconfigurable intelligent surface (RIS) can flexibly control the scattered electro-magnetic (EM) fields for sensing the target and can thus help computational imaging in integrated sensing and communication (ISAC) systems. In this paper, we propose a near-field imaging scheme based on holograghic RIS. To mitigate the inherent ill conditioning of the inverse problem in the imaging system, we design the EM field patterns as masks that help translate the inverse problem into a forward problem. Next, we utilize RIS to generate different virtual EM masks on the target surface and calculate the cross-correlation between the mask patterns and the electric field strength at the receiver. We then provide a RIS design scheme for virtual EM masks by employing a regularization technique. Simulation results demonstrate that the proposed method can achieve high-quality imaging. Moreover, the imaging quality can be improved by generating more virtual EM masks, by increasing the signal-to-noise ratio (SNR) at the receiver, or by placing the target closer to the RIS.

Integrated sensing and communication (ISAC) has opened up numerous game-changing opportunities for future wireless systems. In this paper, we develop a novel scheme that utilizes orthogonal frequency division multiplexing (OFDM) pilot signals in ISAC systems to sense the electromagnetic (EM) property of the target and thus also identify the material of the target. Specifically, we first establish an end-to-end EM propagation model by means of Maxwell equations, where the EM property of the target is captured by a closed-form expression of the ISAC channel, incorporating the Lippmann-Schwinger equation and the method of moments (MOM) for discretization. We then model the relative permittivity and conductivity distribution (RPCD) within a specified detection region. Based on the sensing model, we introduce a multi-frequency-based EM property sensing method by which the RPCD can be reconstructed from compressive sensing techniques that exploits the joint sparsity structure of the EM property vector. To improve the sensing accuracy, we design a beamforming strategy from the communications transmitter based on the Born approximation that can minimize the mutual coherence of the sensing matrix. The optimization problem is cast in terms of the Gram matrix and is solved iteratively to obtain the optimal beamforming matrix. Simulation results demonstrate the efficacy of the proposed method in achieving high-quality RPCD reconstruction and accurate material classification. Furthermore, improvements in RPCD reconstruction quality and material classification accuracy are observed with increased signal-to-noise ratio (SNR) or reduced target-transmitter distance.

Reconfigurable intelligent surface (RIS) can improve the communications between a source and a destination. Recently, continuous aperture RIS is proved to have better communication performance than discrete aperture RIS and has received much attention. However, the conventional continuous aperture RIS is designed to convert the incoming planar waves into the outgoing planar waves, which is not the optimal reflecting scheme when the receiver is not a planar array and is located in the near field of the RIS. In this paper, we consider two types of receivers in the radiating near field of the RIS: (1) when the receiver is equipped with a uniform linear array (ULA), we design RIS coefficient to convert planar waves into cylindrical waves; (2) when the receiver is equipped with a single antenna, we design RIS coefficient to convert planar waves into spherical waves. We then propose the maximum likelihood (ML) method and the focal scanning (FS) method to sense the location of the receiver based on the analytic expression of the reflection coefficient, and derive the corresponding position error bound (PEB). Simulation results demonstrate that the proposed scheme can reduce energy leakage and thus enlarge the channel capacity compared to the conventional scheme. Moreover, the location of the receiver could be accurately sensed by the ML method with large computation complexity or be roughly sensed by the FS method with small computation complexity.

Remote-controlled vascular interventional robots have great potential for use in minimally invasive vascular surgeries in recent years due to their ability to reduce the occupational risk of surgeons and improve the stability and accuracy of surgical procedures. However, blood vessels will suffer from the damage caused by collision with medical instruments to some extent even though the surgeries are very successful. Moreover, when surgeons perform unsafe operations, the unsafe operations will not only seriously affect surgical safety (or even cause serious complications) but also restrict the continuity of operation. In this article, a multilevel concept for operating force is first introduced into surgical procedures as a reference for the choice and design of operation strategies. Based on this concept, a novel multilevel operation strategy is first proposed to reduce blood vessel damage, ensure surgical safety, and allow for continuous operation. This strategy can remind surgeons about the operative conditions in real-time, reduce collision to blood vessels, and eliminate unsafe operations online. A prototype was fabricated and calibrated through calibration experiments and the performance of the multilevel operation strategy was validated throughin vitroandex vivoexperiments. Experimental results demonstrate the engineering effectiveness of the proposed method and motivate the need for furtherin vivostudies to evaluate improvement on surgical safety.

Timely and accurate auxiliary diagnosis of intracranial aneurysm can help radiologist make treatment plans quickly, saving lives and cutting costs at the same time. At present, Digital Subtraction Angiography (DSA) is the gold standard for the diagnosis of intracranial aneurysm, but as radiologists interpret those imaging sequences frame by frame, misdiagnosis might occur. The utilization of computer-aided diagnosis (CAD) can ease the burdens of radiologists and improve the detection accuracy of aneurysms. In this article, a deep learning method is applied to detect the intracranial aneurysm in 3D Rotational Angiography (3D-RA) based on a spatial information fusion (SIF) method, and instead of a 3D vascular model, 2D image sequences are used. Given the intracranial aneurysm and vascular overlap having similar feature in the most time, rather than focusing on distinguishing them in one frame, the morphological differences between frames are considered as major feature. In the training data, consecutive frames of every imaging time series are extracted and concatenated in a specific way, so that the spatial contextual information could be embedded into a single two-dimensional image. This method enables the time series with obvious correlation between frames be directly trained on 2D convolutional neural network (CNN), instead of 3D-CNN with huge computational cost. Finally, we got an accuracy of 98.89%, with sensitivity and specificity of 99.38% and 98.19%, respectively, which proves the feasibility and availability of the SIF feature.