Soyi Jung

dblp:203/9638 · also So-Yi Jung · DBLP profile ↗
← Back
34ranked-venue papers
3as first author
31since 2021 · last 2026
0000-0001-8435-0646ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Computer networks · 20 · 2 first-author · 18 since 2021Systems, architecture and hardware · 6 · 6 since 2021Artificial intelligence and machine learning · 2 · 2 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 2 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021
YearPublicationVenuePosition
2026 Lyapunov-Optimized Traffic Splitting in TN-NTN using QMIX for Enhanced Network Stability
abstract
Next-generation networks aim to deliver ubiquitous services such as enhanced mobile broadband (eMBB) and ultra-reliable low-latency communications (URLLC), where multi-connectivity (MC) leveraging multiple paths is essential. However, terrestrial networks (TNs) alone cannot provide global coverage, motivating the integration of low Earth orbit (LEO)-based non-terrestrial networks (NTNs). This heterogeneous environment introduces complex traffic orchestration challenges, with intelligent control still in its early research phase. To address this, this paper proposes a dynamic traffic splitting framework based on multi-agent reinforcement learning (MARL), where each user equipment (UE) cooperatively learns the optimal traffic ratio using the QMIX algorithm. A Lyapunov-based reward function ensures both QoS satisfaction and long-term stability. Simulation results confirm the framework’s effectiveness in enhancing user experience and network efficiency in integrated TN-NTN systems.
Seoyeong Park, Junyoung Kim 0006, Huiyeon Jang, Soyi Jung
CCNC4
2026 Hybrid Large Language Models and Reinforcement Learning for Energy-Efficient Multisatellite Scheduling: Boosting the Performance From Scratch
abstract
Low Earth Orbit (LEO) satellite constellations are crucial for global connectivity by providing extensive coverage and reduced delays. However, scheduling data transmission in these dynamic networks is challenging due to rapidly changing satellite positions. This research introduces BoostRL (boosted reinforcement learning), a novel framework integrating large language models (LLMs) with reinforcement learning (RL) for efficient scheduling in LEO satellite constellations. BoostRL leverages LLM-generated initial policies to accelerate convergence, thereby guiding the early stages of policy learning while adapting swiftly to dynamic network conditions. It employs a hybrid policy approach which transitions smoothly from LLM recommendations to autonomous RL policies. A tailored initialization of Q-value parameters and an enhanced loss function further optimize learning efficiency, by aligning the initial learning phase with LLM-generated insights. Simulations using two-line element (TLE) orbital data demonstrate that BoostRL achieves rapid convergence and improved efficiency, thereby validating its potential as a scalable, adaptive solution for managing satellite communication networks.
Hyojun Ahn, Gyu Seon Kim, In-Sop Cho, Soyi Jung, Joongheon Kim
IEEE Internet Things J.4
2026 Gateway-Assisted Neural Angular Routing for Hierarchical Satellite Networks
abstract
The increasing demand for ultra-low-latency and high-throughput communication necessitates network architectures capable of ensuring reliable real-time connectivity. To address this requirement, integrated architectures combining terrestrial networks (TN) and non-terrestrial networks (NTN)—particularly hierarchical multi-layer satellite systems composed of geostationary Earth orbit (GEO), medium Earth orbit (MEO), and low Earth orbit (LEO) satellites—have attracted considerable attention. Such hierarchical systems offer global coverage, low latency, and high capacity. However, their highly dynamic topology, fast-moving satellites, and fluctuating link quality pose significant challenges to establishing efficient end-to-end routing paths. Traditional routing methods based on static paths or centralized control lack the adaptability required for such environments. To address these challenges, this paper introduces a neural hierarchical reinforcement learning (HRL)-based routing algorithm for satellite networks. The framework exploits the layered structure of GEO, MEO, and LEO satellites, enabling gateway-assisted routing and adaptive path selection. Control responsibilities are distributed across orbital layers: the GEO layer allocates hop-count budgets, the MEO layer selects feasible LEO paths, and the LEO layer forwards data packets. When inter-satellite links are unavailable, terrestrial gateways provide alternative routes. Routing decisions further account for satellite-to-satellite and satellite-to-ground link quality, with an end-to-end delay-based reward function capturing transmission, processing, and propagation effects. Simulation results show that the proposed neural hierarchical framework, supported by integrated terrestrial and non-terrestrial networking, achieves faster convergence, greater route stability, and better adaptability than conventional routing algorithms and HRL approaches.
Jiseok Jang, In-Sop Cho, Minsu Shin 0001, Joongheon Kim, Soyi Jung
IEEE Internet Things J.5
2026 Multiagent Deep Reinforcement Learning for Joint Movement and User Association of UAV-BS Emergency Indoor User Service
abstract
This paper investigates the joint optimization of unmanned aerial vehicle-mounted base station (UAV-BS) movement and user association for multiple UAV-BSs providing emergency services to indoor users. Specifically, we focus on optimizing associations between UAV-BSs and indoor users within an outdoor-to-indoor path loss model that accounts for floor penetration. The primary objective is to determine the optimal associations between UAV-BSs and indoor users, while also addressing how multiple UAV-BSs should move to establish these associations quickly. To solve this problem, we propose a novel multi-agent reinforcement learning (MARL) architecture featuring three key innovations: a dual-action structure that decouples the complex decision-making process into separate movement and association actions, a multi-agent double deep Q-network (MADDQN) to learn optimal policies, and prioritized experience replay (PER) to improve learning efficiency. Simulation results demonstrate that the proposed algorithm significantly outperforms baseline methods—including a multi-agent deep Q-network (MADQN), multi-agent independent actor-critic (MAIAC), multi-agent deep deterministic policy gradient (MADDPG), and a consensus-based bundle algorithm (CBBA)—across all metrics. Furthermore, a series of rigorous ablation studies systematically validates the contribution of each component. Overall, the simulation results validate the superiority and robustness of our proposed algorithm in dynamic and challenging indoor environments.
Taeyoon Kim 0003, Jihong Park, Junghwa Kang, Jaeyeol Lee, Soyi Jung
IEEE Internet Things J.5
2026 Quantum Multi-Agent Reinforcement Learning for Cooperative Mobile Access in Space-Air-Ground Integrated Networks
abstract
Achieving global space-air-ground integrated network (SAGIN) access only with CubeSats presents significant challenges such as the access sustainability limitations in specific regions (e.g.,polar regions) and the energy efficiency limitations in CubeSats. To tackle these problems, high-altitude long-endurance unmanned aerial vehicles (HALE-UAVs) can complement these CubeSat shortcomings for providing cooperatively global access sustainability and energy efficiency. However, as the number of CubeSats and HALE-UAVs, increases, the scheduling dimension of each ground station (GS) increases. As a result, each GS can fall into the curse of dimensionality, and this challenge becomes one major hurdle for efficient global access. Therefore, this paper provides a quantum multi-agent reinforcement Learning (QMARL)-based method for scheduling between GSs and CubeSats/HALE-UAVs in order to improve global access availability and energy efficiency. The main reason why the QMARL-based scheduler can be beneficial is that the algorithm facilitates a logarithmic-scale reduction in scheduling action dimensions, which is one critical feature as the number of CubeSats and HALE-UAVs expands. Additionally, individual GSs have different traffic demands depending on their locations and characteristics, thus it is essential to provide differentiated access services. The superiority of the proposed scheduler is validated through data-intensive experiments in realistic CubeSat/HALE-UAV settings.
Gyu Seon Kim, Yeryeong Cho, Jaehyun Chung, SooHyun Park, Soyi Jung, Zhu Han 0001, Joongheon Kim
IEEE Trans. Mob. Comput.5
2026 Joint Sustainable Control and Quantum Reinforcement Learning for Energy-Efficient Cube-Satellite Networks
abstract
Satellites have been envisioned as primary non-terrestrial networks capable of seamless global network and surveillance services. Among various satellite types, Cube Satellites (CubeSats) have been actively researched because multiple CubeSats can be conveniently positioned in a target orbit simultaneously and in proximity to Earth. However, CubeSats are small-scale, and thus, they are not able to accommodate a sizable battery, imposing constraints on the duration of their mission. Considering this energy limitation, in order to realize global network services using multiple CubeSats, this paper proposes a novel two-stage Reinforcement Learning (RL) algorithm for energy-efficient CubeSats where RL is utilized for dynamic control under uncertainty. Firstly, sustainable control for single-CubeSat orbital maneuver is considered using deep deterministic policy gradient for vertical position adjustment over a continuous action domain. Secondly, a novel quantum multi-agent RL algorithm for multi-CubeSat cooperative scheduling is designed to realize action dimension reduction into a logarithmic scale based on our proposed Projection-Valued Measure (PVM) over the quantum domain. It is highlighted that our considering two single- and multi-CubeSat problems cannot be separately considered for extreme energy management. The performance evaluation results demonstrate that the proposed algorithm outperforms other benchmarks with 1.51× higher performance in orbital control, 2.71× higher converged reward in enormous action dimensions, and 2.27× higher average network performance.
SooHyun Park, Gyu Seon Kim, Soyi Jung, Zhu Han 0001, Joongheon Kim
IEEE Trans. Mob. Comput.3
2025 Hallucination-Aware Generative Pretrained Transformer for Cooperative Aerial Mobility Control
abstract
This paper proposes SafeGPT, a two-tiered framework that integrates generative pretrained transformers (GPTs) with reinforcement learning (RL) for efficient and reliable un-manned aerial vehicle (UAV) last-mile deliveries. In the proposed design, a Global GPT module assigns high-level tasks such as sector allocation, while an On-Device GPT manages real-time local route planning. An RL-based safety filter monitors each GPT decision and overrides unsafe actions that could lead to battery depletion or duplicate visits, effectively mitigating hallucinations. Furthermore, a dual replay buffer mechanism helps both the GPT modules and the RL agent refine their strategies over time. Simulation results demonstrate that SafeGPT achieves higher delivery success rates compared to a GPT-only baseline, while substantially reducing battery consumption and travel distance. These findings validate the efficacy of combining GPT-based semantic reasoning with formal safety guarantees, contributing a viable solution for robust and energy-efficient UAV logistics.
Hyojun Ahn, Seungcheol Oh, Gyu Seon Kim, Soyi Jung, SooHyun Park, Joongheon Kim
GLOBECOM4
2025 Synergistic Power Optimization in Satellite-Terrestrial Coexistence Networks Via Attention-Driven Deep Reinforcement Learning
abstract
Satellite communication with low Earth orbit (LEO) satellites is a key technology for non-terrestrial networks (NTN) in 6G. However, LEO satellites face challenges such as extensive coverage and interference with terrestrial networks (TN), particularly in NTN-TN coexistence scenarios with ambiguous geographical boundaries. This paper proposes an attentiondriven multi-agent deep Q-network (ATT-MADQN) for dynamic LEO cell transmission power control. By incorporating an attention mechanism optimized for extracting key information, the proposed method enhances learning efficiency and interference management within the MADQN framework. Simulation results demonstrate that ATT-MADQN improves system throughput and coverage probability (CP) while outperforming conventional MADQN in terms of robustness and efficiency.
Junyoung Kim 0006, Jaeyeol Lee, Jiseok Jang, Soyi Jung
VTC2025-Spring5
2025 Optimized Handover Management for Reliable Connectivity in GEO-LEO Satellite Networks via Predictive Reinforcement Learning
abstract
This paper presents a novel methodology to optimize the handover decision of mobile terminals (MTs) within a cooperative geostationary Earth orbit (GEO) and low Earth orbit (LEO) satellite network. Unlike prior research, which primarily focuses on inter-LEO satellite handovers for stationary terminals, this work addresses the handover challenges associated with MTs in a hybrid GEO-LEO satellite architecture. The proposed framework employs a convolutional neural network (CNN)-long-short-term memory (LSTM) encoder-decoder model to accurately predict the reference signal received power (RSRP) of potential handover target satellites, enabling more informed decision-making. A multi-agent double deep Q-network (MADDQN) algorithm is implemented to determine the optimal handover target, leveraging predicted RSRP data while considering factors such as network load and satellite connection duration. The proposed approach minimizes unnecessary handovers, enhances data throughput for MTs, and ensures stable connectivity and quality of service under diverse network conditions. The proposed method contributes to handover optimization in cooperative GEO-LEO satellite networks, offering valuable insights for next-generation satellite communication systems.
Huiyeon Jang, Junyoung Kim 0006, Minsu Shin 0001, In-Sop Cho, Soyi Jung
WiOpt5
2025 Joint Interference Approximation and Guard-Band Management for Spectrum-Efficient Integrated NTN-TN Networks
abstract
The integration of the non-terrestrial network (NTN) and the terrestrial network (TN) aims to provide seamless and ubiquitous connectivity by using satellites, unmanned aerial vehicles, and high-altitude platforms to expand coverage in remote and underserved areas. Despite its potential, the integration of NTN and TN faces significant challenges in interference management and resource allocation, primarily due to the shared spectrum and the distinct operational characteristics of both network types. To address these issues, this paper proposes a reinforcement learning-based framework for dynamic resource allocation and interference management in integrated NTN-TN environments. The framework is designed to maintain balanced performance between NTN and TN while achieving a high signal-to-interference-plus-noise ratio (SINR) and coverage probability (CP). An interference approximation model, validated via Monte Carlo simulations, ensures reliability and accuracy across various scenarios. The simulation results demonstrate that the framework effectively mitigates interference, adapting to varying user equipment densities while minimizing performance degradation. The proposed approach significantly enhances the co-existence of NTN and TN, achieving higher SINR and CP compared to existing methods. These findings establish a robust foundation for scalable and efficient NTN-TN integration in next-generation networks.
Jiseok Jang, Junyoung Kim 0006, Joongheon Kim, Soyi Jung
IEEE Internet Things J.4
2025 Entanglement-Controlled Quantum Federated Learning
abstract
According to the advances in quantum computing and distributed learning, quantum federated learning (QFL) has recently become an emerging field of study. In QFL, each quantum computer or device locally trains its quantum neural network (QNN) with trainable gates, and communicates only these gate parameters over classical channels, without costly quantum communications. To successfully opeate QFL under various and dynamic channel conditions in Internet of Things (IoT) environments, this article develops a novel depth-controllable architecture of entangled slimmable QNNs (eSQNNs), and thus, proposes an entangled slimmable QFL (eSQFL) that communicates the superposition-coded parameters of eSQNNs. Even though the proposed eSQNN-based eSQFL is superior, training the depth-controllable eSQNN architecture is challenging due to high-entanglement entropy and interdepth interference. Therefore, the proposed method in this article mitigates the interference using entanglement controlled universal (CU) gates and an inplace fidelity distillation (IPFD) regularizer penalizing interdepth quantum state differences, respectively. Furthermore, the proposed method optimizes the superposition coding power allocation by deriving and minimizing the convergence bound of eSQFL. The novelty of this work is evaluated via extensive simulations in terms of prediction accuracy, fidelity, and entropy compared to Vanilla QFL as well as under different channel conditions and various data distributions.
SooHyun Park, Hyunsoo Lee 0001, Soyi Jung, Jihong Park, Mehdi Bennis, Joongheon Kim
IEEE Internet Things J.3
2025 Quantum federated learning with pole-angle quantum local training and trainable measurement
SooHyun Park, Hyunsoo Lee 0001, Seok Bin Son, Soyi Jung, Joongheon Kim
Neural Networks4
2025 Adaptive Excavation Automation in Complex Soil Environments Using Reinforcement Learning
abstract
This paper presents an advanced excavation automation framework that improves trajectory efficiency and improves operational stability in diverse and complex soil conditions. The framework integrates the fundamental equation of earthmoving (FEE) with the proximal policy optimization (PPO) algorithm to enable adaptive trajectory planning during excavation. Using these estimates, the PPO algorithm iteratively optimizes excavation strategies to minimize applied force and improve trajectory efficiency, contributing to potential energy savings. The validation of the proposed framework is performed through simulations conducted under varying soil conditions and initial height configurations. The results show an 18.1% reduction in total excavation resistance compared to baseline models, achieving a mean absolute error (MAE) of 0.065 m. These findings confirm the effectiveness of the framework in reducing excavation resistance and improving task precision in simulated environments.
Mingyu Shin, Junhyung Cho, Joongheon Kim, Soyi Jung
IEEE Trans Autom. Sci. Eng.4
2025 Joint scalable quantum convolutional neural network and reverse fidelity training for high-accurate recognition in unmanned aerial vehicle surveillance
Emily Jimin Roh, Joongheon Kim, Soyi Jung, SooHyun Park
J. Supercomput.3
2025 Intelligent Extra Resource Allocation for Cooperative Awareness Message Broadcasting in Cellular-V2X Networks
abstract
According to recent advances in cellular vehicle-to-everything (cellular-V2X) communication networks, vehicle-to-vehicle (V2V) networks can be performed without infrastructure support. The corresponding wireless standard has been actively studied by the 3rd generation partnership project (3GPP) and has been proposed to realize V2V communications via sidelink interfaces. This standard specifically introduces cellular-V2X Mode 4, where individual vehicles access wireless resources in a distributed manner by using sensing-based semi-persistent scheduling (SPS) for avoiding collisions in cooperative awareness messages (CAMs). However, the legacy SPS scheme is able to be challenged by resource scheduling collisions, which obviously degrade performance. To tackle this problem, this paper proposes intelligent adaptive and additive extra resource allocation strategy that responds in real time to estimate traffic density fluctuations, aiming to decrease resource collisions as well as improve utilization performance. Moreover, our additional intelligent solution, which is fundamentally inspired by Lyapunov optimization-based drift-plus-penalty framework, dynamically controls the number of resources to minimize transmission outage probability subject to resource constraints. The corresponding data-intensive performance analysis results verify that the proposed algorithm is able to improve performance compared to the other benchmarks.
Soyi Jung, Joongheon Kim
IEEE Trans. Netw. Serv. Manag.1
2024 Markov Decision Policies for Distributed Angular Routing in LEO Mobile Satellite Constellation Networks
abstract
This article proposes a distributed angular routing algorithm in time-varying dynamic low Earth orbit (LEO) satellite constellation networks. For designing satellite routing algorithms, it is essential to consider 1) distributed operation due to the difficulty in global centralized computation and 2) angle-based computation under the consideration of orbit coordinate systems. Therefore, our proposed routing algorithm is based on distributed angular computation. Moreover, the proposed algorithm is designed by the Markov decision process (MDP) for discrete-time sequential decision making in time-varying LEO satellite networks. As a result, this article proposes an MDP-based distributed angular routing (MDAR) algorithm for seamless LEO routing. Based on the reward formulation in terms of angular differences in MDP formulation, our proposed distributed angular routing algorithm pursues orbit-geometrically straight-line data delivery from the source to its associated destination. Finally, our proposed routing algorithm is evaluated in the realistic environment with real-world satellite data, i.e., two line elements (TLEs), and the results confirm that our proposed algorithm outperforms the others in terms of routing success rate, reward convergence, and successful throughput.
SooHyun Park, Gyu Seon Kim, Soyi Jung, Joongheon Kim
IEEE Internet Things J.3
2024 Joint Quantum Reinforcement Learning and Stabilized Control for Spatio-Temporal Coordination in Metaverse
abstract
In order to build realistic metaverse systems, enabling high synchronization between physical-space and virtual meta-space is essentially required. For this purpose, this paper proposes a novel system-wide coordination algorithm for high synchronization under characteristics (i.e., highly realistic meta-space construction under the constraints of physical-space). The proposed algorithm consists of the following three stages. The first stage is quantum multi-agent reinforcement learning (QMARL)-based scheduling for low-delay temporal-synchronization using differentiated age-of-information (AoI) during data gathering in physical-space by observers for meta-space construction. This is beneficial for scalability according to action dimension reduction in reinforcement learning computation. The second stage is for creating virtual contents under delay constraints in meta-space based on the gathered data. When rendering regions that have received more user attention, avatar-popularity is considered for spatio-synchronization. Thus, a stabilized control mechanism is designed for time-average reality quality maximization for each region. The last stage is for caching based on avatar-popularity and AoI which can be helpful in constructing low-delay realistic meta-space. Furthermore, the concept of AoI is divided into two separate sub-concepts of physical AoI and virtual AoI such that the AoI in virtual meta-space can be thoroughly implemented.
SooHyun Park, Jaehyun Chung, Chanyoung Park 0002, Soyi Jung, Minseok Choi, Sungrae Cho, Joongheon Kim
IEEE Trans. Mob. Comput.4
2024 Cooperative Multi-UAV Positioning for Aerial Internet Service Management: A Multi-Agent Deep Reinforcement Learning Approach
abstract
This paper proposes a novel multi-agent deep reinforcement learning (MADRL)-based positioning algorithm for multiple unmanned aerial vehicles (UAVs) collaboration in mobile access applications where the UAVs work as mobile base stations. The primary objective of the proposed algorithm is to establish reliable mobile access networks for vehicle-to-everything (V2X) communications. This paper jointly considers energy-efficient UAV operation and reliable wireless communication services for realizing robust mobile access services. For the energy-efficient UAV operation, the reward function formulation of our proposed MADRL algorithm contains the features for UAV energy consumption models in order to realize efficient operations. Furthermore, for the reliable wireless communication services, the quality of service (QoS) requirements of individual users are considered as a part of reward function. Furthermore, this paper considers 60, GHz millimeter-wave (mmWave) mobile access for utilizing the benefits of i) ultra-wide-bandwidth for multi-Gbps high-speed communications and ii) high-directional communications for spatial reuse that is obviously good for avoiding interference among densely deployed users. Lastly, the comprehensive and data-intensive performance evaluation of the proposed MADRL-based algorithm for multi-UAV positioning is conducted. The results of these evaluations demonstrate that the proposed algorithm outperforms other existing algorithms.
Joongheon Kim, SooHyun Park, Soyi Jung, Carlos Cordeiro 0001
IEEE Trans. Netw. Serv. Manag.3
2023 Multi-Agent Deep Reinforcement Learning for Efficient Passenger Delivery in Urban Air Mobility
abstract
It has been considered that urban air mobility (UAM), also known as drone-taxi or electrical vertical takeoff and landing (eVTOL), will play a key role in future transportation. By putting UAM into practical future transportation, several benefits can be realized, i.e., (i) the total travel time of passengers can be reduced compared to traditional transportation and (ii) there is no environmental pollution and no special labor costs to operate the system because electric batteries will be used in UAM system. However, there are various dynamic and uncertain factors in the flight environment, i.e., passenger sudden service requests, battery discharge, and collision among UAMs. Therefore, this paper proposes a novel cooperative multiagent deep reinforcement learning (MADRL) algorithm based on centralized training and distributed execution (CTDE) concepts for reliable and efficient passenger delivery in UAM networks. According to the performance evaluation results, we confirm that the proposed algorithm outperforms other existing algorithms in terms of the number of serviced passengers increase (30%) and the waiting time per serviced passenger decrease (26% ).
Chanyoung Park 0002, SooHyun Park, Gyu Seon Kim, Soyi Jung, Joongheon Kim
ICC4
2023 Demo: EQuaTE: Efficient Quantum Train Engine Design and Demonstration for Dynamic Software Analysis
abstract
This paper proposes an efficient quantum train engine (EQuaTE), a novel tool for quantum machine learning software which plots gradient variances to check whether our quantum neural network (QNN) falls into local minima (called barren plateaus in QNN). EQuaTE can be realized via dynamic analysis of the undetermined probabilistic qubit states. Furthermore, the proposed EQuaTE is capable of HCI-based visual feedback such that software engineers can recognize barren plateaus via visualization, allowing the modification of QNN based on this information.
SooHyun Park, Hao Feng 0002, Won Joon Yun, Chanyoung Park 0002, Youn Kyu Lee, Soyi Jung, Joongheon Kim
ICDCS6
2023 Poster: Coordinated Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Swarms in Autonomous Mobile Access Applications
abstract
This paper proposes a novel centralized training and distributed execution (CTDE)-based multi-agent deep reinforcement learning (MADRL) method for multiple unmanned aerial vehicles (UAVs) control in autonomous mobile access applications. For the purpose, a single neural network is utilized in centralized training for cooperation among multiple agents while maximizing the total quality of service (QoS) in mobile access applications.
Chanyoung Park 0002, Haemin Lee, Won Joon Yun, SooHyun Park, Soyi Jung, Joongheon Kim
ICDCS5
2023 Truthful and performance-optimal computation outsourcing for aerial surveillance platforms via learning-based auction
Soyi Jung, David Mohaisen, Joongheon Kim
Comput. Networks1
2023 Quantum Multiagent Actor-Critic Networks for Cooperative Mobile Access in Multi-UAV Systems
abstract
This article proposes a novel algorithm, named quantum multiagent actor–critic networks (QMACN) for autonomously constructing a robust mobile access system employing multiple unmanned aerial vehicles (UAVs). In the context of facilitating collaboration among multiple UAVs, the application of multiagent reinforcement learning (MARL) techniques is regarded as a promising approach. These methods enable UAVs to learn collectively, optimizing their actions within a shared environment, ultimately leading to more efficient cooperative behavior. Furthermore, the principles of quantum computing (QC) are employed in our study to enhance the training process and inference capabilities of the UAVs involved. By leveraging the unique computational advantages of QC, our approach aims to boost the overall effectiveness of the UAV system. However, employing a QC introduces scalability challenges due to the near intermediate-scale quantum (NISQ) limitation associated with qubit usage. The proposed algorithm addresses this issue by implementing a quantum centralized critic, effectively mitigating the constraints imposed by NISQ limitations. Additionally, the advantages of the QMACN with performance improvements in terms of training speed and wireless service quality are verified via various data-intensive evaluations. Furthermore, this article validates that a noise injection scheme can be used for handling environmental uncertainties in order to realize robust mobile access.
Chanyoung Park 0002, Won Joon Yun, Jae Pyoung Kim, Tiago Koketsu Rodrigues, SooHyun Park, Soyi Jung, Joongheon Kim
IEEE Internet Things J.6
2023 Quantum Multiagent Actor-Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management
abstract
As one of the latest fields of interest in both academia and industry, quantum computing has garnered significant attention. Among various topics in quantum computing, variational quantum circuits (VQCs) have been noticed for their ability to carry out quantum deep reinforcement learning (QRL). This article verifies the potential of QRL, which will be further realized by implementing quantum multiagent reinforcement learning (QMARL) from QRL, especially for Internet-connected autonomous multirobot control and coordination in smart factory applications. However, the extension is not straightforward due to the nonstationarity of classical MARL. To cope with this, the centralized training and decentralized execution (CTDE) QMARL framework is proposed under the Internet connection. A smart factory environment with the Internet of Things (IoT)-based multiple agents is used to show the efficacy of the proposed algorithm. The simulation corroborates that the proposed QMARL-based autonomous multirobot control and coordination performs better than the other frameworks.
Won Joon Yun, Jae Pyoung Kim, Soyi Jung, Joongheon Kim
IEEE Internet Things J.3
2023 SlimFL: Federated Learning With Superposition Coding Over Slimmable Neural Networks
abstract
Federated learning (FL) is a key enabler for efficient communication and computing, leveraging devices’ distributed computing capabilities. However, applying FL in practice is challenging due to the local devices’ heterogeneous energy, wireless channel conditions, and non-independently and identically distributed (non-IID) data distributions. To cope with these issues, this paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNN). Integrating FL with SNNs is challenging due to time-varying channel conditions and data distributions. In addition, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, which makes SNN ill-suited for FL. Motivated by this, we propose a communication and energy-efficient SNN-based FL (namedSlimFL) that jointly utilizessuperposition coding (SC)for global model aggregation andsuperposition training (ST)for updating local models. By applying SC, SlimFL exchanges the superposition of multiple-width configurations decoded as many times as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations while avoiding inter-width interference during backpropagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also deals with non-IID data distributions and poor channel conditions, which is also corroborated by data-intensive simulations.
Won Joon Yun, Yunseok Kwak, Hankyul Baek, Soyi Jung, Mingyue Ji, Mehdi Bennis, Jihong Park, Joongheon Kim
IEEE/ACM Trans. Netw.4
2022 Hierarchical Reinforcement Learning using Gaussian Random Trajectory Generation in Autonomous Furniture Assembly
abstract
In this paper, we propose a Gaussian Random Trajectory guided Hierarchical Reinforcement Learning (GRT-HL) method for autonomous furniture assembly. The furniture assembly problem is formulated as a comprehensive human-like long-horizon manipulation task that requires a long-term planning and a sophisticated control. Our proposed model, GRT-HL, draws inspirations from the semi-supervised adversarial autoencoders, and learns latent representations of the position trajectories of the end-effector. The high-level policy generates an optimal trajectory for furniture assembly, considering the structural limitations of the robotic agents. Given the trajectory drawn from the high-level policy, the low-level policy makes a plan and controls the end-effector. We first evaluate the performance of GRT-HL compared to the state-of-the-art reinforcement learning methods in furniture assembly tasks. We demonstrate that GRT-HL successfully solves the long-horizon problem with extremely sparse rewards by generating the trajectory for planning.
Won Joon Yun, David Mohaisen, Soyi Jung, Jong-Kook Kim, Joongheon Kim
CIKM3
2022 Quality-Aware Real-Time Augmented Reality Visualization under Delay Constraints
abstract
Augmented reality (AR) is one of emerging applications in modern multimedia systems research. Due to intensive time-consuming computations for AR visualization in mobile devices, quality-aware real-time computing under delay constraints is essentially required. Inspired by Lyapunov optimization framework, this paper proposes a time-average quality maximization method for the AR visualization under delay considerations.
Rhoan Lee, SooHyun Park, Soyi Jung, Joongheon Kim
ICDCS3
2022 AoI-Aware Markov Decision Policies for Caching
abstract
We consider a scenario that utilizes road side units (RSUs) as distributed caches in connected vehicular networks. The goal of the use of caches in our scenario is for rapidly providing contents to connected vehicles under various traffic conditions. During this operation, due to the rapidly changed road environment and user mobility, the concept of age-of-information (AoI) is considered for (1) updating the cached information as well as (2) maintaining the freshness of cached information. The frequent updates of cached information maintain the freshness of the information at the expense of network resources. Here, the frequent updates increase the number of data transmissions between RSUs and MBS; and thus, it increases system costs, consequently. Therefore, the tradeoff exists between the AoI of cached information and the system costs. Based on this observation, the proposed algorithm in this paper aims at the system cost reduction which is fundamentally required for content delivery while minimizing the content AoI, based on Markov Decision Process (MDP) and Lyapunov optimization.
SooHyun Park, Soyi Jung, Minseok Choi, Joongheon Kim
ICDCS2
2022 Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design
abstract
In recent years, quantum computing (QC) has been getting a lot of attention from industry and academia. Especially, among various QC research topics, variational quantum circuit (VQC) enables quantum deep reinforcement learning (QRL). Many studies of QRL have shown that the QRL is superior to the classical reinforcement learning (RL) methods under the constraints of the number of training parameters. This paper extends and demonstrates the QRL to quantum multi-agent RL (QMARL). However, the extension of QRL to QMARL is not straightforward due to the challenge of the noise intermediate-scale quantum (NISQ) and the non-stationary properties in classical multi-agent RL (MARL). Therefore, this paper proposes the centralized training and decentralized execution (CTDE) QMARL framework by designing novel VQCs for the framework to cope with these issues. To corroborate the QMARL framework, this paper conducts the QMARL demonstration in a single-hop environment where edge agents offload packets to clouds. The extensive demonstration shows that the proposed QMARL framework enhances 57.7% of total reward than classical frameworks.
Won Joon Yun, Yunseok Kwak, Jae Pyoung Kim, Hyunhee Cho, Soyi Jung, Jihong Park, Joongheon Kim
ICDCS5
2022 Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks
abstract
This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) architectures. FL preserves data privacy by exchanging the locally trained models of mobile devices. By adopting SNNs as local models, FL can flexibly cope with the time-varying energy capacities of mobile devices. Combining FL and SNNs is however non-trivial, particularly under wireless connections with time-varying channel conditions. Furthermore, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, so are ill-suited to FL. Motivated by this, we propose a communication and energy efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple width configurations that are decoded as many as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations, while avoiding the inter-width interference during back propagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also can counteract non-IID data distributions and poor channel conditions, which is also corroborated by simulations.
Hankyul Baek, Won Joon Yun, Yunseok Kwak, Soyi Jung, Mingyue Ji, Mehdi Bennis, Jihong Park, Joongheon Kim
INFOCOM4
2022 Cooperative Multiagent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control
abstract
CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments.This article creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimizing overlapping and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This article develops a multiagent deep reinforcement learning-based management scheme for reliable industry surveillance in smart city applications. The core idea this article employs is autonomously replenishing the UAV's deficient network requirements with communications. Via intensive simulations, our proposed algorithm outperforms the state-of-the-art algorithms in terms of surveillance coverage, user support capability, and computational costs.
Won Joon Yun, SooHyun Park, Joongheon Kim, Myungjae Shin, Soyi Jung, David Mohaisen
IEEE Trans. Ind. Informatics5
2019 Reducing Consecutive Collisions in Sensing Based Semi Persistent Scheduling for Cellular-V2X
abstract
Cellular-vehicle to everything (Cellular-V2X) has been designed to support Long Term Evolution (LTE) communication standard, whereas direct short range communication (DSRC) is based on IEEE 802.11p. In this paper, we focus on cellular-V2X with Mode 4, where vehicles schedule their resources in a distributed way employing sensing based semi persistent scheduling (SPS) to avoid collision of cooperative awareness message (CAM). We analyze the effect of Mode 4 configuration and some parameters on the performance. We find that reselection counter is one of the key parameters that has an effect on consecutive collisions. Therefore, we propose a resource alternative selection (RAS) algorithm based on SPS, which reserves and allocates the multi-resources alternatively during the period of reselection counter. The simulation results show that the proposed RAS algorithm can significantly improve performance, which can not only increase message reception reliability (MRR) but also decrease consecutive message collisions.
Soyi Jung, Hye-Rim Cheon
VTC Fall1
2018 Adaptive resource allocation and congestion control algorithm for massive devices in LTE-A
abstract
Internet of things (IoT) devices in the long term evolution-advanced (LTE-A) network require the random access (RA) for the data transmission. The base station in LTE-A requires a decision algorithm for the number of preambles and for the probability of devices to enter contention. This paper proposes an adaptive resource allocation and congestion control algorithm referring to the most recently observed contention results. Furthermore, this paper also proposes an estimation method which estimates the number of contending devices and the number of activated devices in the network based on the unused number of preambles. The performance evaluation using the RA simulator shows that the proposed algorithm can achieve the throughput which is close to the optimal throughput. We also address the limit of current LTE-A system to support massive number of devices based on the evaluation results.
Sung-Hyung Lee, Soyi Jung
WCNC2
2017 Performance evaluation of random access response estimation scheme for IoT communications
abstract
This paper analyzed the performance of the LTE random access (RA). We describes a RA performance analysis simulator for LTE to support massive number of devices. To evaluate the performance, we developed new modules to simulate LTE's RA procedures because there is a limit when it accommodate massive number of devices in existing LTE module. This paper also represents performance analysis results according to the number of preambles and maximum number of preamble transmissions in the same environment with 3GPP TR 37.868. With this results, we proposed the random access response (RAR) estimation scheme. As a result, the performance of RA is greatly improved compared to conventional RA.
Seung-Su Yoo, Sung-Hyung Lee, Soyi Jung
ICC3