Reinforcement learning (RL) has shown significant promise in sequential portfolio optimization. A typical solution involves optimizing cumulative returns using historical offline data. However, it may produce less generalizable policies that merely ''memorize'' optimal buying and selling actions from the offline data while neglecting the non-stationary nature of the financial market. We frame portfolio optimization of stock data as a specific type of offline RL problem. Our method, MetaTrader, presents two key contributions. First, it introduces a novel bilevel RL algorithm that operates on both the original stock data and its transformations. The core idea is that a robust policy should generalize effectively to out-of-distribution data. Second, we propose a new temporal difference (TD) method that leverages a transformation-based conservative TD target to address value overestimation under limited offline data. Empirical results on two publicly available datasets demonstrate that MetaTrader outperforms existing methods, including both traditional stock prediction models and RL-based trading approaches.

This paper presents a deformable spherical robot with a six-strut topological structure capable of achieving multimodal locomotion in complex amphibious environments. The robot realizes isotropic rolling and asymmetric jumping through its innovative geometric-based configuration while integrating an airbag-driven module for underwater buoyancy control. Based on collision dynamics analysis, we develop a prototype of the deformable spherical robot. Experiments conducted on land, in transition zones, and underwater validate the robot’s multimodal locomotion feasibility in multi-medium environments.

Learning object-centric representations from unsupervised videos is challenging. Unlike most previous approaches that focus on decomposing 2D images, we present a 3D generative model named DynaVol-S for dynamic scenes that enables object-centric learning within a differentiable volume rendering framework. The key idea is to perform object-centric voxelization to capture the 3D nature of the scene, which infers per-object occupancy probabilities at individual spatial locations. These voxel features evolve through a canonical-space deformation function and are optimized in an inverse rendering pipeline with a compositional NeRF. Additionally, our approach integrates 2D semantic features to create 3D semantic grids, representing the scene through multiple disentangled voxel grids. DynaVol-S significantly outperforms existing models in both novel view synthesis and unsupervised decomposition tasks for dynamic scenes. By jointly considering geometric structures and semantic features, it effectively addresses challenging real-world scenarios involving complex object interactions. Furthermore, once trained, the explicitly meaningful voxel features enable additional capabilities that 2D scene decomposition methods cannot achieve, such as novel scene generation through editing geometric shapes or manipulating the motion trajectories of objects.

In federated learning (FL), the servers are generally seen as omnipotent to distribute the full models to all clients. This is not the case in wireless systems. The broadcast of models inevitably excludes some users with poor channel gain but valuable computation power and diverse local data. In this paper, taking the various computation and communication capabilities of users into account, we propose a federated learning scheme incorporating unstructured model pruning and non-orthogonal transmission. The pruning process divides the model into the core and the extended parts, with the ratio adaptive to the users’ achievable working rates. With non-orthogonal transmission, the strong users get both parts of the model while the weak get only the core part. Simulation results show that the proposed scheme involves more users than traditional wireless FL and improves the accuracy of tasks within the given communication rounds.

Unsupervised learning of object-centric representations in dynamic visual scenes is challenging. Unlike most previous approaches that learn to decompose 2D images, we present DynaVol, a 3D scene generative model that unifies geometric structures and object-centric learning in a differentiable volume rendering framework. The key idea is to perform object-centric voxelization to capture the 3D nature of the scene, which infers the probability distribution over objects at individual spatial locations. These voxel features evolve over time through a canonical-space deformation function, forming the basis for global representation learning via slot attention. The voxel features and global features are complementary and are both leveraged by a compositional NeRF decoder for volume rendering. DynaVol remarkably outperforms existing approaches for unsupervised dynamic scene decomposition. Once trained, the explicitly meaningful voxel features enable additional capabilities that 2D scene decomposition methods cannot achieve: it is possible to freely edit the geometric shapes or manipulate the motion trajectories of the objects.

With the development of visible light communication technology, visible light indoor positioning technology has become one of the research hotspots in indoor positioning technology. Visible light indoor positioning uses the light source information in the scene to locate the receiving end. However, the indoor environment is complex, and there are likely to be mirrors, glass and other reflective objects. These objects reflect to generate a pseudo-light source that transmits information similar to the actual light source, affecting the positioning result. Aiming at the recognition of pseudo-light sources, this paper adds classification steps to the algorithm based on chiral light source array, and proposes two methods that can improve the accuracy of identifying true and false light sources: K-Nearest Neighbor (KNN) classification and Weighted K-Nearest Neighbor (WKNN) classification and correction. Based on the chiral detection algorithm of vector product, we compare the success rates of the improved methods with the algorithm without the improved methods. The simulation results show that the accuracy of false detection can be further improved by adding the improved method, so as to improve the positioning accuracy.

Directional Antennas have drawn some attention in flying ad-hoc networks (FANET). Most of the existing works consider only beam steering in a 2-dimensional plane. However, when the networks scale up to a coverage of hundreds of kilometers, some challenges emerge for the 2D methods. In this paper, we design a 3D beam scanning pattern and propose an efficient scanning strategy for the neighbor discovery process based on the analysis regarding the distance between nodes as well as the earth's curvature. Also, we fine-tune the pitch angle of the beam according to the scanner's altitude. Simulation results show that the proposed scheme outperforms the conventional scheme in terms of neighbor discovery time.

Typical RL-for-finance solutions directly optimize trading policies over the noisy market data, such as stock prices and trading volumes, without explicitly considering the future trends and correlations of different investment assets as we humans do. In this paper, we present StockFormer, a hybrid trading machine that integrates the forward modeling capabilities of predictive coding with the advantages of RL agents in policy flexibility. The predictive coding part consists of three Transformer branches with modified structures, which respectively extract effective latent states of long-/short-term future dynamics and asset relations. The RL agent adaptively fuses these states and then executes an actor-critic algorithm in the unified state space. The entire model is jointly trained by propagating the critic's gradients back to the predictive coding module. StockFormer significantly outperforms existing approaches across three publicly available financial datasets in terms of portfolio returns and Sharpe ratios.

Predictive learning ideally builds the world model of physical processes in one or more given environments. Typical setups assume that we can collect data from all environments at all times. In practice, however, different prediction tasks may arrive sequentially so that the environments may change persistently throughout the training procedure. Can we develop predictive learning algorithms that can deal with more realistic, non-stationary physical environments? In this paper, we study a new continual learning problem in the context of video prediction, and observe that most existing methods suffer from severe catastrophic forgetting in this setup. To tackle this problem, we propose the continual predictive learning (CPL) approach, which learns a mixture world model via predictive experience replay and performs test-time adaptation with non-parametric task inference. We construct two new benchmarks based on RoboNet and KTH, in which different tasks correspond to different physical robotic environments or human actions. Our approach is shown to effectively mitigate forgetting and remarkably outperform the naíve combinations of previous art in video prediction and continual learning.

Cyber adversaries have increasingly leveraged social engineering attacks to breach large organizations and threaten the well-being of today's online users. One clever technique, the “watering hole” attack, compromises a legitimate website to execute drive-by download attacks by redirecting users to another malicious domain. We introduce a game-theoretic model that captures the salient aspects for an organization protecting itself from a watering hole attack by altering the environment information in web traffic so as to deceive the attackers. Our main contributions are (1) a novel Social Engineering Deception (SED) game model that features a continuous action set for the attacker, (2) an in-depth analysis of the SED model to identify computationally feasible real-world cases, and (3) the CyberTWEAK algorithm which solves for the optimal protection policy. To illustrate the potential use of our framework, we built a browser extension based on our algorithms which is now publicly available online. The CyberTWEAK extension will be vital to the continued development and deployment of countermeasures for social engineering.