Solving real-world complex tasks using reinforcement learning (RL) without high-fidelity simulation environments or large amounts of offline data can be quite challenging. Online RL agents trained in imperfect simulation environments can suffer from severe sim-to-real issues. Offline RL approaches although bypass the need for simulators, often pose demanding requirements on the size and quality of the offline datasets. The recently emerged hybrid offline-and-online RL provides an attractive framework that enables joint use of limited offline data and imperfect simulator for transferable policy learning. In this paper, we develop a new algorithm, called$\mathrm{H} 2 \mathrm{O}+$, which offers great flexibility to bridge various choices of offline and online learning methods, while also accounting for dynamics gaps between the real and simulation environments. Through extensive simulation and real-world robotics experiments, we demonstrate superior performance and flexibility of$\mathbf{H 2 O}+$over advanced cross-domain online and offline RL algorithms.

With the deepening exploitation of marine resources, the application of underwater communication is becoming increasingly widespread, and the noise in the underwater channel will severely impact the reliability of underwater communication. Therefore, researchers have focused on improving the accuracy and reliability of underwater communication through denoising. Currently, denoising methods are widely used based on algorithms, such as Fourier transform, wavelet transform, convolutional neural network, and time-space domain filtering. Still, they all have some areas for improvement in the application of underwater communication. In this regard, inspired by quantum mechanical tools and adaptive algorithms, this article proposes a quantum adaptive chunk denoising algorithm applied in underwater communication. First, this article proposes an adaptive chunking algorithm to dynamically adjust the chunk size according to the local signal characteristics to capture the signal details better. Second, the chunked signal and image are taken as potentials in the discrete Schrödinger equation and solved for the eigenvalues and eigenvectors of the discrete Hamiltonian matrix constructed by the adaptive chunking algorithm. Finally, the solved values are subjected to total variational denoising and reconstruction after denoising. The algorithm proposed in this article is compared with wavelet denoising based on hard and soft thresholding, Denoising Algorithm Based on an ideal low-pass adaptive filter, sparsity-assisted signal smoothing (SASS) algorithm and Quantum Mechanics Adaptive Basis Denoising Algorithm. The experimental results show that the quantum adaptive chunking denoising algorithm proposed in this article can effectively remove underwater noise from signals and images under the condition of low-signal-to-noise ratio, which improves the reliability and signal quality of underwater communication and meets the denoising standard of underwater communication.

When visiting an online platform, a user generates various actions, such as clicks, long views, likes, comments, etc.To capture user preferences in these aspects, we learn these objectives and return multiple rankings of candidate items for each user.We need to aggregate them into one to truncate the candidate set, and ranking ensemble model is proposed for this task.However, there is a critical issue: though we input abundant information, what model learns depends on the supervision.Unfortunately, the existing supervision is poorly designed, leading to serious information loss issue.To address this issue, we designed an unsupervised loss to compel the ranking ensemble model to learn all information of input rankings, including sequential and numerical information.(1) For sequential information, we design a distance measure between two rankings, and train the ensemble ranking to have similar order with all input rankings by minimizing the distance.(2) For numerical information, we design a decoder to reconstruct values of original rankings from the hidden layer of the model, to guarantee that the model captures as much input information as possible.Our unsupervised loss is compatible with all ranking ensemble models.We optimize several widely-used structures to propose unsupervised ranking ensemble models.We devise comprehensive experiments on two real-world datasets to demonstrate the effectiveness of the proposed models.We also apply our model in a short video platform with billions of users, and achieve significant improvement.