The converging trends of reinforcement learning (RL) control and cloud-fog automation in industrial cyber-physical systems impose multiple challenges for communications to cope with stringent requirements in latency, reliability, control effectiveness and bifurcating user demands. Progressive goal-oriented (GO) communication is a promising technology to tackle the above challenges. This paper takes a two-step approach to design the first progressive codec of GO communications tailored for RL control tasks. The first step is to design a variable-rate coding scheme that extends the boundaries of rate regimes. This step is achieved by empowering the hierarchical variational autoencoder (HVAE) framework with novel algorithms such as mutual information based soft state abstraction (MISA). The second step is to transform variable-rate encoding into progressive encoding. This is achieved by applying residual-based encoding techniques upon latent representations learned by deep neural networks. Experiments on the Cartpole Swingup task demonstrate that the proposed progressive codec can facilitate smooth transitions from the ultra-low rate regime to regular rate regime, while achieving the state-of-the-art performance in terms of rate-distortion-effectiveness tradeoff.

Partially Observable Markov Decision Process (POMDP) is a general framework for a wide range of control tasks, which can benefit from enabling semantic communicatons among different agents. Semantic communications aim to exchange compact messages that can convey task-relevant information between agents. A critical problem in semantic communication is source representation learning, which is governed by a fundamental tradeoff between compactness and sufficiency. Such a tradeoff is still underinvestigated in the context of POMDP. In this paper, we propose HVRL - Hierarchical Variational autoencoders for Reinforcement Learning. Experiments show that our method can effectively balance the pursuit of compactness and sufficiency, thereby learning enough information for decision and mitigates the risk of over-abstraction in the observation space. This approach effectively encodes the endogenous semantic information about the observation itself, and shows good sample efficiency and control performance.