In this paper, we consider a latency-sensitive wireless network and aim at minimizing the overall transmission latency via an optimal cross-layer design. In particular, we assume a packet divided into multiple subpackets is supposed to be routed from a source node to a destination node through a wirelessly connected multi-device network. Each activated routing link is assigned a dedicated subcarrier, allowing simultaneous transmission and reception. Taking into account the routing ability at the network layer and the finite blocklength (FBL) effects at the physical layer, via an error propagation method, we first derive out the average transmission latency for completing a data forwarding task under buffer limit at each routing device, while retransmissions are scheduled against transmission failures. Afterwards, we formulate an average transmission latency minimization problem via jointly optimizing the routing path at the network layer and the blocklength allocation at the physical layer. To optimally address the cross-layer mixed-integer nonlinear problem, we characterize the optimal blocklength design for given routing path as an equation system, which is efficiently solved via iterative fixpoint checks. The performed characterization enables a filtering criterion for efficiently evaluating the performance bound of any routing path with respect to a threshold, based on which we propose an efficient algorithm for the optimal routing path filtering, together with a low-complexity iterative routing algorithm for suboptimal routing design. The global optimal joint solution is obtained as the filtered optimal path, combined with the correspondingly optimized blocklength solution. Finally, we numerically validate the effectiveness and optimality of our proposed solution, as well as the necessity of cross-layer design for latency minimization.

In this paper, we consider a multi-hop full-duplex (FD) relaying system that supports low-latency communication, and aim to explore the potential of FD technology in suppressing transmission latency. Specifically, we begin with a two-hop relaying system, where a source node is expected to transmit a large message to the destination node via a relaying node operating in FD mode. We assume the large message is equally divided into multiple smaller packets, while the whole transmission is operated in a packet-by-packet manner and retransmissions are scheduled against decoding failures. Notably, we have for the first time characterized the expected transmission latency while taking into account the finite blocklength (FBL) impact on transmission reliability. Through a proposed error probability propagation policy, we have recursively derived the expected number of transmissions required to successfully conveying the entire message via FD relaying system. An optimization problem is then formulated to minimize the expected transmission latency by jointly optimizing packet division, blocklength allocation, and transmit power control. To deal with the inherent nonconvexity of the problem, we reformulate it using variable substitution and subsequently construct a tight convex approximation based on an arbitrary feasible point. This facilitates an iterative algorithm that progressively refines the solution until convergence to a suboptimal point. The whole approach for latency characterization and minimization is then extended to the multi-hop relaying scenario. Finally, simulation results validate the convergence behaviours of our proposed algorithms and highlight the latency benefits of our solution compared to both half-duplex relaying and full-duplex relaying without optimal power control.

This paper explores the potential of employing two-hop full-duplex (FD) relaying systems to alleviate transmission latency. The approach involves dividing a message into smaller packets and transmitting them sequentially with possible retransmissions. Notably, we characterize the expected transmission latency of multiple packet transmissions for the first time. By introducing a novel error probability propagation method, the expected number of time slots needed for successfully transmitting all packets is recursively derived. The article tackles the minimization of transmission latency by jointly considering packet division, blocklength per packet, and power allocation. To cope with the complex nonconvex nature of this optimization problem, a subproblem is extracted, and a reformulation utilizing variable substitution is proposed. Furthermore, a tight convex approximation at any feasible point is developed to facilitate the design of an iterative algorithm to gradually converge towards a suboptimal solution. Simulation results validate the efficacy of the proposed solution, demonstrating its convergence and latency advantages over both half-duplex (HD) and FD relaying systems lacking power control.

Deep neural networks (DNNs) lack the precise semantics and definitive probabilistic interpretation of probabilistic graphical models (PGMs). In this paper, we propose an innovative solution by constructing infinite tree-structured PGMs that correspond exactly to neural networks. Our research reveals that DNNs, during forward propagation, indeed perform approximations of PGM inference that are precise in this alternative PGM structure. Not only does our research complement existing studies that describe neural networks as kernel machines or infinite-sized Gaussian processes, it also elucidates a more direct approximation that DNNs make to exact inference in PGMs. Potential benefits include improved pedagogy and interpretation of DNNs, and algorithms that can merge the strengths of PGMs and DNNs.

In this paper, we focus on an ultra-reliable low latency communication (URLLC) scenario, where the access point (AP) is supposed to support latency-critical communication via a non-orthogonal multiple access (NOMA) scheme. Moreover, we allow the device with the stronger channel acting as a relay for cooperatively enhancing the transmission reliability for the other device. Based on the considered NOMA-assisted cooperative scheme, we characterize out the maximum error probability between two devices as the objective to be minimized. Together with an energy constraint for the whole transmission period, we formulate a problem jointly optimizing the blocklength assigned to two phases, i.e., the NOMA phase and the cooperative phase, and power resources allocated in each transmission attempt. To address this non-convex problem, we reformulate the problem by introducing auxiliary variables and construct a tight convex approximation at any feasible local point, based on which we further propose an efficient algorithm for iteratively improving the local point until a convergence to a sub-optimum. Via numerical results, we validate the convergence of the proposed iterative algorithm and confirm the reliability advantages of NOMA-assisted cooperative scheme, compared with multiple benchmarks.

In this paper, towards enabling massive connectivity in the next generation ultra-reliable low latency communication (URLLC) Internet-of-Things (IoT) network, we investigate a scalable clustered network, where the user scheduling at the access point (AP) is completely replaced by the cooperation scheduling among clustered IoT users, in order to alleviate the overload at AP. In particular, while serving the clustered network, the AP simply broadcasts out all data for the whole network. Each clustered user attempts to decode the broadcast signal. Afterwards, cooperation retransmissions will be scheduled among users for compensating the overall transmission reliability. Considering limited energy and blocklength resources, we start with the cooperation case based on a cluster head and aim at fairly minimizing the maximum error probability among all users, while the resource allocation and cooperation scheduling are jointly designed. To deal with the inherent nonconvexity, we construct a tight convex approximation for the problem based on an arbitrary feasible point, which enables an iterative algorithm for constantly improving the solution until a convergence to a suboptimal. Next, to further exploit the high cooperation flexibility in clustered URLLC network, we extend the whole design to the case allowing arbitrary cooperation among users, i.e., the case without cluster head. Finally, simulation results validate the convergence of our proposed algorithms and highlight the reliability benefits over benchmarks. The impact of cluster head selection and the high cooperation flexibility of the case without cluster head are also illustrated.

Although reinforcement learning (RL) has achieved great success in robotic manipulation skills learning, it is still challenging for long-horizon tasks. Combining RL with demonstrations is an effective solution. In this paper, we propose a novel hierarchical learning from demonstrations method for long-horizon tasks, which leverages (i) object-centered segmentation of demonstrations to automatically segment the teaching trajectories into episodes. (ii) a bi-level hierarchical imitation learning method with a parallel training mechanism to train the two-level policies simultaneously. Experimental results on three challenging long-horizon tasks with sparse rewards show that our proposed method significantly outperforms state-of-art approaches in terms of both sample-efficiency and success rate. Moreover, our method is the only one which achieves satisfactory performance in tasks of multi-object stack and multi-object push&stack.

Exploration in environments with sparse rewards remains a challenging problem in Deep Reinforcement Learning (DRL). For the off-policy method, it usually needs a large number of training samples. With the growing dimensions of state and action space, this method becomes more and more sample-inefficient. In this paper, we propose a novel fast exploration method for off-policy reinforcement learning, called Density-based Intrinsic Motivation and Self-adaptive Action Noise (DIMSAN). Our main contribution is twofold: (1) We propose a Density-based Intrinsic Motivation (DIM) method. It introduces a new intrinsic-reward generation mechanism based on samples’ density estimation during experience replay and encourages the agent to seek novel and unfamiliar states. (2) We propose a Self-adaptive Action Noise (SAN) to deal with the exploration-exploitation tradeoffs, which could automatically change the exploration step through adding adaptive action space noise. The synergy between DIM and SAN could guide the agent to search the state and action space with high efficiency. We evaluate our method on the benchmark manipulation tasks and the designed challenging ones. Empirical results show that our method outperforms the existing methods in terms of convergence speed and sample efficiency, especially in challenging tasks.

Exploration in environments with sparse feed-back remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience Replay (ACDER), which leverages (i) a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully and (ii) the dynamic initial states selection as an automatic exploratory curriculum to further improve the sample-efficiency. Our approach complements Hindsight Experience Replay (HER) by introducing a new way to pursue valuable states. Experiments conducted on four challenging robotic manipulation tasks with binary rewards, including Reach, Push, Pick&Place and Multi-step Push. The empirical results show that our proposed method significantly outperforms existing methods in the first three basic tasks and also achieves satisfactory performance in multi-step robotic task learning.