Existing offline hierarchical reinforcement learning methods rely on high-level policy learning to generate subgoal sequences. However, their efficiency degrades as task horizons increase, and they lack effective strategies for stitching useful state transitions across different trajectories. We propose Graph-Assisted Stitching (GAS), a novel framework that formulates subgoal selection as a graph search problem rather than learning an explicit high-level policy. By embedding states into a Temporal Distance Representation (TDR) space, GAS clusters semantically similar states from different trajectories into unified graph nodes, enabling efficient transition stitching. A shortest-path algorithm is then applied to select subgoal sequences within the graph, while a low-level policy learns to reach the subgoals. To improve graph quality, we introduce the Temporal Efficiency (TE) metric, which filters out noisy or inefficient transition states, significantly enhancing task performance. GAS outperforms prior offline HRL methods across locomotion, navigation, and manipulation tasks. Notably, in the most stitching-critical task, it achieves a score of 88.3, dramatically surpassing the previous state-of-the-art score of 1.0. Our source code is available at: https://github.com/qortmdgh4141/GAS.

Hierarchical Reinforcement Learning (HRL) is specially designed for environments characterized by long-term goals and sparse rewards. High-level policies in HRL learn to generate appropriate subgoals aimed at accomplishing the final goal, while low-level policies focus on achieving these designated subgoals. Recently, graph-based HRL algorithms have demonstrated enhanced learning capabilities through the structural representation of state spaces as graphs. However, existing graph-based HRL methods still often generate inefficient subgoals. This paper introduces a new method, Novelty-aware Graph Traversal and Expansion (NGTE), which selects an optimal node at the graph boundary, termed an Outpost Subgoal, as a direct path toward the final goal. Once the Outpost Subgoal is reached, NGTE transitions into an exploration phase, offering exploration subgoals within a reachable distance to efficiently expand the graph. Demonstrated in complex environments such as quadruped robot navigation and robotic arm manipulation, NGTE consistently outperforms existing graph and non-graph HRL methods, showing outstanding performance, especially in the most challenging scenarios with fixed start and fixed goal conditions.

In the field of Multi-Person Pose Estimation (MPPE), Radio Frequency (RF)-based methods can operate effectively regardless of lighting conditions and obscured line-of-sight situations. Existing RF-based MPPE methods typically involve either 1) converting RF signals into heatmap images through complex preprocessing, or 2) applying a deep embedding network directly to raw RF signals. The first approach, while delivering decent performance, is computationally intensive and time-consuming. The second method, though simpler in preprocessing, results in lower MPPE accuracy and generalization performance. This paper proposes an efficient and lightweight one-stage MPPE model based on raw RF signals. By sub-grouping RF signals and embedding them using a shared single-layer CNN followed by multi-head attention, this model outperforms previous methods that embed all signals at once through a large and deep CNN. Additionally, we propose a new self-supervised learning (SSL) method that takes inputs from both one unmasked subgroup and the remaining masked subgroups to predict the latent representations of the masked data. Empirical results demonstrate that our model improves MPPE accuracy by up to 15 in [email protected] compared to previous methods using raw RF signals. Especially, the proposed SSL method has shown to significantly enhance performance improvements when placed in new locations or in front of obstacles at RF antennas, contributing to greater performance gains as the number of people increases. Our code and dataset is open at Github.

Model-based reinforcement learning (MBRL) has been used to efficiently solve vision-based control tasks in high-dimensional image observations. Although recent MBRL algorithms perform well in trained observations, they fail when faced with visual distractions in observations. These task-irrelevant distractions (e.g., clouds, shadows, and light) may be constantly present in real-world scenarios. In this study, we propose a novel self-supervised method, Dream to Generalize (Dr. G), for zero-shot MBRL. Dr. G trains its encoder and world model with dual contrastive learning which efficiently captures task-relevant features among multi-view data augmentations. We also introduce a recurrent state inverse dynamics model that helps the world model to better understand the temporal structure. The proposed methods can enhance the robustness of the world model against visual distractions. To evaluate the generalization performance, we first train Dr. G on simple backgrounds and then test it on complex natural video backgrounds in the DeepMind Control suite, and the randomizing environments in Robosuite. Dr. G yields a performance improvement of 117% and 14% over prior works, respectively. Our code is open-sourced and available at https://github.com/JeongsooHa/DrG.git

Reinforcement learning (RL) has achieved considerable success in many fields, but applying it to real-world problems can be costly and risky because it requires a lot of online interaction. Recently, offline RL has shown the possibility of extracting a solution through existing logged data without online interaction. In this work, we propose an offline hierarchical RL method, Guider (Guide to Control), that can efficiently solve long-horizon and sparse-reward tasks from offline data. The high-level policy sequentially generates a subgoal that can guide the agent to arrive at the final goal, and the lower-level policy learns how to reach each given guided subgoal. In the process of learning from offline data, the key is to make the low-level policy reachable to the generated subgoals. We show that high-quality subgoal generation is possible through pre-training a latent subgoal prior model. The well-regulated subgoal generation improves performance while avoiding distributional shifts in offline RL by breaking down long, complex tasks into shorter, easier ones. For evaluations, Guider outperforms prior offline RL methods in long-horizon robot navigation and complex manipulation benchmarks. Our code is available at https://github.com/gckor/Guider.

We design a novel learning-based spectrum sensing model. Under the insight that an autocorrelation curve yields richer information than a single sum of received signal powers for detecting the presence of a primary user, we propose a convolutional neural network-based deep learning model, called deep spectrum sensing (DSS), that receives an autocorrelation curve as input. Extensive simulation results show that our DSS model has a higher performance than existing deep-learning-based models that use raw signals or spectrograms as an input. Furthermore, DSS can be trained with much smaller amounts of data than the existing models, and is a lighter model compared with the existing models. Finally, we evaluate the effectiveness of the DSS implementation over a real testbed consisting of universal software radio peripheral and GNU radio packages. The experimental results are consistent with the simulation performance.

Mobile communication technology is evolving rapidly and becoming increasingly ubiquitous, thereby increasing the demand for uplink data-intensive applications (e.g., personal broadcasting and live augmented/virtual reality videos). Recently, to facilitate a cost-effective and smooth transition from 4G to 5G networks, most carriers leverage existing 4G infrastructures using a dual connectivity (DC) feature. DC increases uplink throughput and mobility robustness; however, it also causes unprecedented dynamic fluctuations in radio channels due to the coverage discrepancy between 4G and 5G networks. Thus, in this article, we propose a self-attention-based deep learning model to predict uplink radio resources in 5G DC. We trained the proposed model on commercial 5G DC traffic data from three major carriers in South Korea and obtained an average prediction accuracy of 95.08% under various mobility and cell-load conditions. The proposed model explains the rationale for the obtained predictions by highlighting the parts of the input time-series data that are important to realize accurate prediction. We also demonstrate the usability of the proposed model using a network emulator based on real-world 5G trace data. Extensive evaluations demonstrate that the existing congestion control algorithms can achieve excellent performance when used with the proposed model.

Vision-based reinforcement learning requires efficient and robust representations of image-based observations, especially when the images contain distracting (task-irrelevant) elements such as shadows, clouds, and light. It becomes more important if those distractions are not exposed during training. We design a Self-Predictive Dynamics (SPD) method to extract task-relevant features efficiently, even in unseen observations after training. SPD uses weak and strong augmentations in parallel, and learns representations by predicting inverse and forward transitions across the two-way augmented versions. In a set of MuJoCo visual control tasks and an autonomous driving task (CARLA), SPD outperforms previous studies in complex observations, and significantly improves the generalization performance for unseen observations. Our code is available at https://github.com/unigary/SPD.

This paper investigates the impact of large flows in Content Centric Networks (CCN). In CCN, routers have caches, and store all the data after forwarding. If large flows temporarily occupy a content cache, they may evict popular chunks from the cache, and it results in low cache hit ratio. We mathematically analyzed the amount of occupancy of a large flow in a cache, and realized that a few large flows can constitute a significant portion of a cache. Our simulation results showed that small flows experience cache hit ratio degradation as the number of large flows increases. Finally, we present that limiting the occupancy of large flows in a cache can effectively improve the cache hit ratios for small flows.

We present a scalable and topologically-aware application-layer multicast approach, specially designed for large-scale distributed applications. The proposed approach constructs topologically-aware data paths which are based on topological clustering of multicast group members. The approach does not require any exact network topology information, but instead requires the relative location information of members using landmarks. We partition the members into topologically-aware clusters based on the ordering of their close landmarks. We hierarchically arrange the clusters and separate data paths into two types (i.e., inside-cluster path and outside-cluster path) to exclude outsider nodes, not belonging to the same cluster, from the inside-cluster paths. Our results on performance evaluation show that constructing topologically-aware data paths can reduce unnecessary high latency and redundant network resource usage with low overhead over existing scalable approaches.