Jun Liu 0014

dblp:95/3736-14 · DBLP profile ↗
← Back
39ranked-venue papers
4as first author
27since 2021 · last 2026
0000-0003-4007-6109ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Computer networks · 19 · 3 first-author · 15 since 2021Graphics, computer vision, multimedia, augmented reality and games · 4Applied, interdisciplinary, general and emerging computing · 4 · 4 since 2021Artificial intelligence and machine learning · 3 · 3 since 2021
YearPublicationVenuePosition
2026 Workload-Aware Routing Optimization via Graph Reinforcement Learning for Joint Delay Minimization in Edge-Cloud Networks
abstract
The rapid proliferation of end devices and their hosted applications has significantly increased the demand for data processing. However, lightweight edge endpoints often suffer from limited computational capabilities due to power and battery constraints. Consequently, computation-intensive tasks are typically offloaded to cloud servers located in data centers for processing. While existing studies often abstract the edge–cloud connection as a direct link, the actual backbone network features complex topologies and constrained bandwidth, which intensify as workload scales up. To address this challenge, this paper investigates routing optimization within the backbone network under task offloading scenarios, and proposes a workload-aware deep reinforcement learning (DRL)-based routing algorithm for joint minimization of transmission and processing delays. Specifically, we first develop a detailed system model using discrete event simulation. Then, we design an integrated DRL-based decision-making scheme that simultaneously determines optimal routing paths and target data centers. Extensive experiments on diverse real-world network topologies and heterogeneous task workloads demonstrate that the proposed algorithm significantly outperforms conventional baselines in reducing both transmission and processing delays.
Yuqian Song, Jingli Zhou, Shudan Yu, Jun Liu 0014
IEEE Internet Things J.4
2026 ST-VA-AR: Learning velocity-aware action representations with mixture of spatiotemporal attention
Jiangning Wei, Ke Li 0004, Lan Yang 0014, Dandan Xiao, Jun Liu 0014
Pattern Recognit.6
2026 Adaptive Sacrifice for QoS-Aware Routing: A Graph Reinforcement Learning Approach
abstract
The Internet today hosts a multitude of communication sessions from diverse vertical industries, each with distinct and increasingly stringent quality-of-service (QoS) requirements across multiple performance metrics. However, QoS-aware routing remains a significant challenge in traffic engineering, as existing solutions struggle to adapt to dynamic network conditions and meet these rigorous QoS demands. To address this issue, this paper proposes a multi-agent graph reinforcement learning-based routing algorithm that provides differentiated treatment for multiple services. First, we explore both nodebased and link-based graph reinforcement learning paradigms for performance comparison. Second, two key mechanisms, i.e., a packet sacrifice mechanism and a Tchebycheff-based reward function, are designed to realize adaptive sacrifice behavior patterns, aiming to optimize the lower bound of service satisfaction rates and enhance fairness across services. Furthermore, to ensure practical applicability, we devise a distributed computing architecture featuring neighborhood-restricted data acquisition and asynchronous historical information retrieval. Extensive simulation results demonstrate that our proposed algorithms significantly outperform benchmark methods regarding the minimum service satisfaction rate, even under unseen networks. Besides, the distributed computing architecture is proven to incur no performance penalty, which can be generalized to other resource-constrained applications.
Yuqian Song, Jingli Zhou, Shudan Yu, Jun Liu 0014
IEEE Trans. Mob. Comput.4
2025 Constraint-Aware Probabilistic Packet Forwarding Based on Deep Reinforcement Learning
Guocheng Lin, Yang Xiao 0013, Jun Liu 0014
Networking3
2025 PVD-TD3: A Latency-Oriented Multi-Agent Reinforcement Learning Algorithm for Multipath Routing in DetNet
Yang Xiao 0013, Jun Liu 0014
Networking3
2025 Enabling Adaptive Optimization of Energy Efficiency and Quality of Service in NR-V2X Communications via Multiagent Deep Reinforcement Learning
abstract
The Third Generation Partnership Project has standardized new-radio vehicle-to-everything (NR-V2X) to facilitate advanced use cases for safety-critical message conveyance. However, there is a paucity of resource allocation research for high-performance NR-V2X communications. In this article, we investigate an adaptive optimization issue of energy efficiency (EE) and Quality of Service (QoS) in NR-V2X networks inspired by the Tchebycheff function. To address this issue, we first formulate the resource allocation task as a time-variant mixed-integer nonlinear programming (MINLP) problem. Then, we propose a fully decentralized multiagent deep reinforcement learning (MADRL)-based algorithm characterized by a multitask-actor shared-critic (MTA-SC) architecture and a localized training and distributed execution (LTDE) framework to promote efficient learning and minimize information exchange. Finally, we implement the proposed algorithm in a three-lane highway NR-V2X scenario. Numerical results demonstrate that the proposed algorithm comprehensively outperforms benchmarks in terms of convergence performance, scalability, and robustness.
Yuqian Song, Yang Xiao 0013, Jun Liu 0014
IEEE Internet Things J.3
2025 Adaptive Joint Routing and Caching in Knowledge-Defined Networking: An Actor-Critic Deep Reinforcement Learning Approach
abstract
By integrating the software-defined networking (SDN) architecture with the machine learning-based knowledge plane, knowledge-defined networking (KDN) is revolutionizing established traffic engineering (TE) methodologies. This paper investigates the challenging joint routing and caching problem in KDN-based networks, managing multiple traffic flows to improve long-term quality-of-service (QoS) performance. This challenge is formulated as a computationally expensive non-convex mixed-integer non-linear programming (MINLP) problem, which exceeds the capacity of heuristic methods to achieve near-optimal solutions. To address this issue, we present DRL-JRC, an actor-critic deep reinforcement learning (DRL) algorithm for adaptive joint routing and caching in KDN-based networks. DRL-JRC orchestrates the optimization of multiple QoS metrics, including end-to-end delay, packet loss rate, load balancing index, and hop count. During offline training, DRL-JRC employs proximal policy optimization (PPO) to smooth the policy optimization process. In addition, the learned policy can be seamlessly integrated with conventional caching solutions during online execution. Extensive experiments demonstrate the comprehensive superiority of DRL-JRC over baseline methods in various scenarios. Meanwhile, DRL-JRC consistently outperforms the heuristic baseline under partial policy deployment during execution. Compared to the average performance of the baseline methods, DRL-JRC reduces the end-to-end delay by 51.14% and the packet loss rate by 40.78%.
Yang Xiao 0013, Huihan Yu, Yixing Wang, Jun Liu 0014, Nirwan Ansari
IEEE Trans. Mob. Comput.5
2024 Link2Link: A Robust Probabilistic Routing Algorithm via Edge-centric Graph Reinforcement Learning
abstract
As network services become more complex, efficient routing has become crucial for ensuring end-user satisfaction. To address this challenge, researchers are increasingly turning to routing algorithms that integrate Graph Neural Networks (GNNs) with Deep Reinforcement Learning (DRL), leveraging the natural graph structure of network topologies. However, a significant challenge with existing algorithms is their inability to generalize across different topologies without requiring retraining, a constraint that is impractical in real-world applications. To overcome this limitation, we propose a novel GNN-DRL-based routing algorithm, Link2Link, designed to decouple DRL-learned knowledge from specific network topologies by focusing on link-level features. Extensive experiments demonstrate that Link2Link achieves robust performance across diverse topologies, consistently outperforming OSPF without requiring retraining, making it a scalable and adaptable solution for modern network routing challenges.
Jingli Zhou, Yuqian Song, Jun Liu 0014
CNSM5
2024 PPO-TEGN: Towards Robust Deep Reinforcement Learning-based Routing Algorithm
abstract
The reliability of routing algorithms is critical for network operations. Deep Reinforcement Learning (DRL)-based routing algorithms are emerging as promising solutions for achieving next-generation intelligent network routing. However, existing DRL-based routing algorithms struggle to handle topological changes such as link failures due to the restriction of traditional neural networks. In this paper, we propose a novel routing algorithm named PPO-TEGN for enhancing robustness against topological changes. PPO-TEGN leverages Proximal Policy Optimization (PPO) framework for training, and incorporates a meticulously designed Transformer-based Edge-Enhanced Graph Neural Network (TEGN) to extract graph information for probabilistic routing scenarios. Additionally, motivated by the analysis of traffic distribution, we propose a subgraph-enhancement mechanism to improve the ability of perceiving potential congestion at the source and destination nodes. We conduct experiments on simulated and real-world topologies in different traffic patterns. Experimental results reveal that PPO-TEGN outperforms baselines including the traditional routing algorithm and DRL-based routing algorithms with basic GNN in reducing end-to-end (E2E) delay in networks with link failures, implying the strong robustness of the proposed algorithm.
Junze Li, Ke Yu 0001, Jun Liu 0014
GLOBECOM4
2024 CGTR: Leveraging Contrastive Learning and Graph Transformer for Deep Reinforcement Learning Based Robust Routing
abstract
As a crucial role in communication networks, the routing algorithm determines how to transmit traffic from sources to destinations. In recent years, Deep Reinforcement Learning (D RL) has been introduced into routing algorithms to address dynamic traffic demands in complex networks. However, most DRL-based routing algorithms are implemented by traditional neural networks, which can only handle fixed-size matrices and operate on a fixed topology. Fortunately, Graph Neural Network (GNN) has been proposed to process graph-structured data and generalize on different graphs. Furthermore, contrastive learning has also been successfully applied to DRL for decision-making in multiple environments. In this paper, we introduce GNN and contrastive learning to DRL-based routing algorithms, and propose a Contrastive Graph Transformer Routing (CGTR) algorithm to improve the robustness of routing against link fail-ures. Aiming at probabilistic packet routing scenarios, we design Edge-enhanced Graph Transformer and Contrastive Grouping Routing in CG TR, enabling it to perceive unseen link failures without retraining. To evaluate the robustness of CG TR, we conduct extensive experiments with different patterns of traffic demands on both generated and real-world network topologies. The experimental results demonstrate that CGTR outperforms all baselines on unseen topologies with link failures, highlighting its stronger robustness.
Junze Li, Yang Xiao 0013, Sixu Liu, Jun Liu 0014
ICC5
2024 MV-MOE: A Visual Mixture-of-Experts Model for Optical-SAR Image Matching
abstract
Optical and Synthetic Aperture Radar (SAR) matching produces spatial and semantic correspondences of the input images, playing a pivotal role in the registration process. However, due to the difference in radiation characteristics and geometric properties, even the same target may manifest distinctive morphological and feature expressions in the cross-modal images. Consistent feature extraction remains a challenge for optical-SAR image matching. Therefore, based on the salient image patterns (keypoint, line, and block), a Visual Mixture-of-Experts method for optical-SAR image Matching (MV-MOE) is proposed. It facilitates adaptive graphical representation of multi-modal images across various scenes through the multi-task learning framework. With the aid of the attention mechanism, the task-related basic features are reconstructed into matching-related features, yielding similarity along with spatial offset vectors. Additionally, we employ a multi-level feature extraction backbone based on the visual retentive block, enhancing local feature perception with the preservation capability of the recurrent network structure. Experiments demonstrate the advantages of the proposed method on multi-modal image matching and its contribution to the subsequent registration.
Jingyi Cao, Yanan You, Jun Liu 0014
IGARSS3
2024 TSK: A Trustworthy Semantic Keypoint Detector for Remote Sensing Images
abstract
Keypoint detection aims to automatically locate the most significant and informative points in remote sensing images (RSIs), which directly affects the accuracy of matching and registration. In contrast to the handcrafted keypoint detectors that heavily rely on the morphological gradient of corner, line, and ridge, the learning-based detectors emphasize obtaining reliable keypoints from deep features. However, the limited accuracy of semantics undermines the reliability of keypoints, especially in challenging scenarios characterized by repeated textures and boundaries. Therefore, a novel trustworthy semantic keypoint (TSK) detector is proposed for RSIs. It utilizes a lightweight multiscale feature extraction and fusion network, along with a saliency keypoint localization mechanism, to facilitate keypoint detection. Notably, the TSK detector employed explicit semantics, which is refined with multiple learning strategies about repeatability and representability across the multigranularity reasoning spaces, namely, pixel window, neighbor window, and existence entity. Finally, several metrics about repeatability, matching, and registration are used to evaluate the performance of the TSK detector and other competitive methods. Four RSI datasets, including MICGE, HRSCD, OSCD, and SZTAKI, are used to verify performances. TSK detector achieves competitive performance against existing methods.
Jingyi Cao, Yanan You, Jun Liu 0014
IEEE Trans. Geosci. Remote. Sens.4
2024 Scalable QoS-Aware Multipath Routing in Hybrid Knowledge-Defined Networking With Multiagent Deep Reinforcement Learning
abstract
Multipath routing remains a challenging issue in traffic engineering (TE) as existing solutions are incapable of handling the evolving network dynamics and stringent quality-of-service (QoS) requirements. To address it, multi-agent deep reinforcement learning (MADRL) is a promising technique that provides more elaborate multipath routing strategies. However, prevalent MADRL-based solutions still suffer inapplicability as they fail to ensure both scalability and QoS awareness. In this paper, we leverage the emerging hybrid knowledge-defined networking (KDN) architecture, and propose a collaborative MADRL-based multipath routing algorithm. Two novel mechanisms, i.e., parallel agent replication and periodic policy synchronization, are devised for agent design to ensure the practicality of the proposed method. In addition, an efficient communication mechanism is established to facilitate multi-agent collaboration by enabling scalable observation and reward exchange. Featuring a multi-agent twin-actor-critic (MA-TAC) learning structure and a proximal policy optimization (PPO) -based training process, the proposed algorithm consists of alternately scheduled execution and training phases for practical deployment. We compare the performance of our proposed method with those of several benchmark methods. Extensive simulation results demonstrate that the proposed method achieves significantly better scalability, QoS awareness, and stability than the benchmark methods under various environment settings.
Yang Xiao 0013, Huihan Yu, Jun Liu 0014
IEEE Trans. Mob. Comput.4
2024 Collaborative Multi-Agent Deep Reinforcement Learning for Energy-Efficient Resource Allocation in Heterogeneous Mobile Edge Computing Networks
abstract
Mobile edge computing (MEC) is an enabling technology for next-generation network architectures to deliver more diversified communication services and meet more demanding quality-of-service (QoS) requirements. However, owing to the growing scale and heterogeneity of the networks, energy-efficient resource allocation in heterogeneous MEC (Het-MEC) networks faces great challenges. As an emerging research area, multi-agent deep reinforcement learning (MADRL) is expected to realize autonomous resource allocation in Het-MEC networks by learning from trial and error. Nevertheless, existing MADRL-based solutions are usually not applicable to practical scenarios by the limitations of centralized control schemes or massive signaling overhead. To address the issue, we first formulate the energy-efficient resource allocation problem in Het-MEC networks as a time-variant mixed-integer nonlinear programming (MINLP) problem. Then, we propose a fully decentralized collaborative MADRL-based algorithm featuring the multi-actor shared-critic (MASC) architecture and the regional training distributed execution (RTDE) framework, which effectively stabilizes model training and reduces information exchange, respectively. Finally, we conduct extensive simulations to evaluate the performance of the proposed algorithm. Numerical results demonstrate that the proposed algorithm comprehensively outperforms several mainstream baseline methods in terms of convergence performance, scalability, and robustness.
Yang Xiao 0013, Yuqian Song, Jun Liu 0014
IEEE Trans. Wirel. Commun.3
2023 Deep Reinforcement Learning Based Probabilistic Cognitive Routing: An Empirical Study with OMNeT++ and P4
abstract
This paper presents an empirical study on deep reinforcement learning (DRL) based probabilistic cognitive routing using the OMNeT++ framework and programming protocol-independent packet processors (P4). The proposed algorithm combines the power of DRL and cognitive routing to achieve efficient and adaptive probabilistic routing in software-defined networking (SDN) environments. To facilitate the research, we develop a dedicated network simulation environment using the OMNeT++ framework and a self-developed SDN platform based on P4. The empirical study highlights the importance of a comprehensive training and validation process in both simulation and real-world SDN environments. Through closed-loop training, the cognitive routing framework provides real-time feedback from the actual network environment to the simulation environment, allowing the agent to excel in real-world network environments. Meanwhile, the results demonstrate that solely testing the algorithm in either environment is inadequate for evaluating its performance accurately.
Yixing Wang, Yang Xiao 0013, Yuqian Song, Jingli Zhou, Jun Liu 0014
CNSM5
2023 Deep Reinforcement Learning Based Dynamic Routing Optimization for Delay-Sensitive Applications
abstract
With the rapid development of the Internet and the approaching of the next-generation networking, the number and variety of delay-sensitive applications have increased dramatically. Nowadays, how to properly route delay-sensitive packets in complex network environment and meet the stringent quality-of-service (QoS) requirements of delay-sensitive applications remains a great challenge. Towards this end, this paper proposes a deep reinforcement learning (DRL)-based routing algorithm for delay-sensitive applications featuring the proximal policy optimization (PPO) method and the front-convergent actor-critic network (FCACN) technique. To meet the high demand of delay-sensitive applications, we consider the packet survival time (ST) to help our algorithm perform better and make up for the shortage of the time-to-live (TTL) mechanism in IP network. We conduct extensive experiments to prove the efficiency and reliability of the proposed algorithm. Experimental results show that the proposed algorithm outperforms two traditional routing protocols and two state-of-the-art DRL-based routing algorithms in terms of minimizing delay and packet loss rate.
Yang Xiao 0013, Guocheng Lin, Gang He 0008, Fang Liu 0026, Jun Liu 0014
GLOBECOM7
2023 CloudLoc-NeRF: Point-cloud Assisted Volume Location for Neural Radiance Fields
abstract
Realistic rendering results are available to be generated by volume-based neural rendering methods like NeRF. However, the existing schemes take the color vector as the unique supervision information, which leads to ambiguous prediction results of the volume density in the same spatial position through different rendering rays. This problem is common in large-scale scene reconstruction based on remote sensing images taken by UAVs and other equipment. To this end, we contrive to integrate spatial point cloud and multi-view image information, making the sparse point cloud the calibration for key features and regions. Therefore, the CloudLoc-NeRF is proposed. For volume information extraction, the multi-resolution hash coding and voxel are adopted to estimate the district for ray marching and extract volume features efficiently. For the point cloud, annular sampling and plane coding are used to combine image features of the training views and the point cloud. The regions with high feature response in multiple modal data should correspond to the regions with high volume density. In addition, an optimization method based on point cloud density is proposed. The weight parameter of volume density confidence is constructed to symbolize the correlation between density distribution and point cloud density. We verified the performance of our method on NVSF and the wide-area scene reconstruction dataset. Experiments showed that CloudLoc-NeRF accurately expresses the details of the rendered scene and produces better view synthesis results.
Jingyi Cao, Yanan You, Songzhi Gao, Jun Liu 0014
IGARSS4
2023 GAPPO - A Graph Attention Reinforcement Learning based Robust Routing Algorithm
abstract
Routing algorithms, which determine how to deliver traffic from the source to the destination, are essential for next-generation networks and the internet. To make optimal routing decisions in complex network environments, researchers have leveraged Deep Reinforcement Learning (DRL) to design next-generation routing mechanisms. However, most existing DRL-based routing algorithms are implemented using traditional Neural Networks (NN), which lack robustness against topology changes. In this paper, we propose a novel algorithm called GAPPO, which integrates Graph Attention Network (GAT) and Proximal Policy Optimization (PPO) to optimize routing policies with the objective of minimizing end-to-end (E2E) latency in networks affected by link failures. To evaluate the performance of the proposed algorithm, we conduct a series of experiments on dynamically changing topologies with link failures under different loads. Experimental results demonstrate that GAPPO outperforms benchmark algorithms in both simulated and real-world networks, confirming its powerful robustness against link failures.
Yang Xiao 0013, Sixu Liu, Xucong Lu, Fang Liu 0026, Jun Liu 0014
PIMRC7
2023 Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things
abstract
As a promising technology in the 5G era, the artificial intelligence (AI) enabled Internet of controllable things (IoCT) is expected to be an integral part of heterogeneous networks (HetNets) in the future. However, the realization of ultra-reliable low-latency communications (URLLC) in IoCT communications underlaid HetNet has stringent quality of service (QoS) requirements, resulting in unprecedented challenges for existing wireless resource allocation methods. In this paper, we first describe a cellular HetNet model with uplink IoCT communications, then formulate a dynamic mixed-integer nonlinear programming (MINLP) resource allocation problem for maximizing the long-term average energy efficiency under URLLC requirements including reliability, latency, and transmission rate. To solve the problem, we propose a decentralized MADRL-based resource allocation algorithm with a decentralized partially observable Markov decision process (dec-POMDP) and a mixed-centralized-decentralized (MCD) framework to address the partial observability and the scalability issues, respectively. In addition, we design a reward function featuring the objective decomposition, baseline-guided scaling, and QoS violation penalty so that the agents are coordinated. Extensive experiments demonstrate the convergence, scalability, and robustness of the proposed algorithm. Besides, the proposed algorithm substantially outperforms conventional resource allocation methods and different agent communication mechanisms in terms of maximizing energy efficiency.
Yang Xiao 0013, Yuqian Song, Jun Liu 0014
IEEE Trans. Wirel. Commun.3
2022 Towards Energy Efficient Resource Allocation: When Green Mobile Edge Computing Meets Multi-Agent Deep Reinforcement Learning
abstract
Mobile edge computing (MEC) extends the computing power to the edge of communication networks, which has been considered as a promising technology to further improve the quality of communication services in the near future. Nevertheless, the issue of MEC-empowered energy efficient resource allocation has not been well studied. To maximize the longterm energy efficiency for green MEC-enabled heterogeneous networks (HetNets), we proposed a decentralized multi-agent deep reinforcement learning (MADRL) resource allocation algorithm. Based on the proximal policy optimization (PPO) framework, our proposed algorithm enables observation exchange to coordinate the policies of multiple agents. Simulation results show that our proposed algorithm significantly outperforms three baseline methods in terms of effectiveness, robustness, and scalability.
Yang Xiao 0013, Yuqian Song, Jun Liu 0014
ICC3
2022 Make Object Connect: A Pose Estimation Network for UAV Images of the Outdoor Scene
abstract
As the basics of 3D vision, pose estimation with 2D images is of significance in 3D reconstruction, UAV positioning, and other fields. However, the related works focus on the natural images and pay less attention to the wide-coverage UAV remote sensing (RS) images. In fact, the relationship between objects in UAV images can benefit pose estimation. Therefore, aiming at the outdoor scene captured by the UAV monocular camera, a novel pose estimation network that emphasizes the association between objects is proposed. The multi-scale visual features extracted by the convolutional neural network (CNN) are manipulated by the object-agnostic segmentation model to indicate the existing space of all possible objects in the whole scene. The features of all possible objects are embedded into vectors, and then processed with a graph convolution network (GCN) for relationship analysis. Based on the known sparse point cloud and the optimized features of 2D images, the camera pose is regressed iteratively by 3D visual geometry. To verify the feasibility of the network, experiments are conducted on the Extended CMU Seasons and the simulation UAV dataset. Results prove that our network emphasizes more features on the small objects and obtains superior pose estimation results.
Jingyi Cao, Yanan You, Le Xia, Jun Liu 0014
IGARSS4
2022 Attentive Dual-Head Spatial-Temporal Generative Adversarial Networks for Crowd Flow Generation
abstract
Crowd flow generation, which aims to simulate the flows of crowd in the future is of great importance to many real-life applications including epidemic spreading and traffic management. The challenges of accurate crowd flow generation come from both the complex spatial-temporal correlations of crowd flow data and the small amount of training data, which is largely not well studied and addressed in existing works. In this paper, we propose a novel attentive dual-head spatial-temporal generative adversarial network entitled ADST-GAN to simulate multi-step crowd flow. To augment the learning power of the generator, we adopt attentive temporal queue and self attention mechanism to automatically capture the complex global spatial-temporal dependencies of crowd flow data. As for the discriminator, we design a dual-head architecture with two-objective training to avoid the negative effects of quick overfitting. To evaluate the effectiveness of the proposed method, we conduct extensive experiments over the bike and taxi trip datasets in New York. The results demonstrate the proposed method outperforms seven state-of-the-art baselines significantly in terms of the quality of simulated crowd flow data.
Jianxue Li, Yang Xiao 0013, Jiawei Wu 0004, Yaozhi Chen, Jun Liu 0014
PIMRC5
2022 Deep Reinforcement Learning Enabled Energy-Efficient Resource Allocation in Energy Harvesting Aided V2X Communication
abstract
With the commercialization of the 5th generation mobile networks, vehicle-to-everything (V2X) communication has gained tremendous attention over the last decade. However, prevailing research has not sufficiently deliberated on the energy efficiency (EE) optimization issue. This paper proposes a decentralized multi-agent deep reinforcement learning (DRL) based resource allocation algorithm. Moreover, we leverage energy harvesting (EH) to achieve long-term EE maximization. Based on the proximal policy optimization (PPO) framework, we invoke power splitting (PS) to divide the harvested energy delicately. Numerical results demonstrate that our proposed algorithm outperforms traditional and straightforward DRL-based resource allocation approaches in effectiveness and robustness.
Yuqian Song, Yang Xiao 0013, Yaozhi Chen, Jun Liu 0014
PIMRC5
2022 Deep Reinforcement Learning Based Beamforming for Throughput Maximization in Ultra-Dense Networks
abstract
Ultra-dense network (UDN) is a promising technology for 5G and beyond communication systems to meet the requirements of explosive data traffic. However, the dense distribution of wireless terminals potentially leads to severe interference and deteriorate network performance. To address this issue, beamforming is widely used to coordinate the interference in UDNs and improve receive gains by controlling the phase of multiple antennas. In this paper, we propose a multi-agent deep reinforcement learning (DRL) based beamforming algorithm to achieve more dynamic and fast beamforming adjustment. In the proposed algorithm, the agents inside beamforming controllers are distributively trained while exchanging partial channel state information (CSI) for better optimizing beamforming vectors to achieve maximized throughputs in UDNs. The evaluation results demonstrate that the proposed algorithm significantly improves the computation efficiency, as well as achieves the highest network throughput compared to several baselines.
Huihan Yu, Yang Xiao 0013, Jiawei Wu 0004, Fang Liu 0026, Jun Liu 0014
WCNC6
2022 Dual-branch network via pseudo-label training for thyroid nodule detection in ultrasound image
Ruoning Song, Chuang Zhu, Long Zhang 0020, Yihao Luo, Jun Liu 0014, Jie Yang 0023
Appl. Intell.6
2021 Power Allocation for Device-to-Multi-Device Enabled HetNets: A Deep Reinforcement Learning Approach
abstract
Device-to-device ($D$2D) communication exploits the geographical proximity by allowing neighboring devices to di-rectly communicate with each other, which becomes one of the most promising technologies to improve the spectral and energy efficiency for 5G and beyond communication systems. To further improve the spectral efficiency and generalize ap-plication scenarios, the emerging device-to-multi-device (D2M$D$) communication enables the D2D transmitter to communicate with multiple receivers simultaneously. In this paper, we consider a heterogeneous network (HetNet) where multiple D2M$D$clusters coexist with the base station (BS) and cellular users (CUs). All D2MD clusters share the same downlink channel as the cellular network, which potentially leads to severe co-channel interfer-ence. To solve this problem, we leverage the deep reinforcement learning (DRL) and propose the deep reinforcement power allocation (DRPA) algorithm to dynamically allocate power for D2MD communication in HetNets. In addition, we apply the centralized training distributed execution (CTDE) technique to accelerate the training process and improve the robustness of DRPA. Simulation results demonstrate that the DRPA algorithm outperforms baseline methods in terms of maximizing the average sum-rate. In addition, the DRPA algorithm is robust to the changes of network environment while achieving near-optimal performance.
Yang Xiao 0013, Jiawei Wu 0004, Jun Liu 0014
GLOBECOM3
2021 Multi-level colonoscopy malignant tissue detection with adversarial CAC-UNet
Chuang Zhu, Ke Mei, Yihao Luo, Jun Liu 0014, Ying Wang 0043, Mulan Jin
Neurocomputing5
2020 Cross-Stained Segmentation from Renal Biopsy Images Using Multi-Level Adversarial Learning
abstract
Segmentation from renal pathological images is a key step in automatic analyzing the renal histological characteristics. However, the performance of models varies significantly in different types of stained datasets due to the appearance variations. In this paper, we design a robust and flexible model for cross-stained segmentation. It is a novel multi-level deep adversarial network architecture that consists of three sub-networks: (i) a segmentation network; (ii) a pair of multi-level mirrored discriminators for guiding the segmentation network to extract domain-invariant features; (iii) a shape discriminator that is utilized to further identify the output of the segmentation network and the ground truth. Experimental results on glomeruli segmentation from renal biopsy images indicate that our network is able to improve segmentation performance on target type of stained images and use unlabeled data to achieve similar accuracy to labeled data. In addition, this method can be easily applied to other tasks.
Ke Mei, Chuang Zhu, Jun Liu 0014, Yuanyuan Qiao 0002
ICASSP4
2020 Privacy preserving distributed data mining based on secure multi-party computation
Jun Liu 0014, Yang Xiao 0013, Nirwan Ansari
Comput. Commun.1
2018 CTSD: A Dataset for Traffic Sign Recognition in Complex Real-World Images
abstract
Traffic sign recognition (TSR) is an indispensable component for vision-based system of self-driving car. Promising results have been achieved which especially benefit from the rapid development of deep neural networks recently. However, there are few works focusing on the algorithms’ performances towards different complex conditions, such as weather and viewpoint variations. In this paper, we propose a new real-world TSR dataset, which is a dataset with several fine-grained conditions fine labeled involving weather, light condition, occlusion, distance, color fading and camera angle. Detailed and unbiased comparison results are reported about the performances of several state-of-the-arts on our proposed and five public TSR datasets. Experimental results demonstrate that current arts for TSR are still far from satisfactory especially when it comes to complex real-world cases.
Yanting Zhang 0001, Yonggang Qi, Jun Liu 0014, Jie Yang 0023
VCIP4
2018 Revealing connectivity structural patterns among web objects based on co-clustering of bipartite request dependency graph
Jun Liu 0014, Nirwan Ansari
Wirel. Networks2
2017 VACA: A high-performance variable-length adaptive CRC algorithm
abstract
Cyclic Redundancy Check (CRC) is one of the most important error-detecting codes used in digital communication and storage systems. A number of algorithms based on parallel table lookups, which can process 32 or 64 bits at a time, have been proposed to accelerate CRC generation process. In recent years, parallel CRC algorithms with multi-processor architecture become popular, which further increase the processing speed. However, there is still much room to reduce the synchronization and recombination overheads for large messages to be processed. In this paper, we propose a coarse-grained parallel CRC algorithm for efficient n-core processor implementation. Our algorithm can be easily combined with existing fine-grained CRC methods to obtain a high performance. The synchronization and recombination in the proposed algorithm are deferred and needed only once to minimize the overhead cost. The evaluation results demonstrate that the proposed algorithm can achieve a speedup of a factor of almost n.
Mucong Chi, Jun Liu 0014
PIMRC2
2017 Accurate quantile estimation for skewed data streams
abstract
Quantile estimation is a fundamental method to generate descriptions of the distribution of data for data management and analysis. Although the investigation and design of efficient quantile estimation algorithm has attracted much study, the problem of accurately finding quantiles in the case of skewed data streams, which are prevalent in many data sources like IP traffic streams in the 4G mobile network and text data, is still not well addressed. In this paper we specifically address the problem of estimating the quantiles of massive skewed data streams by designing and implementing an incremental quantile estimation with nonlinear-interpolation algorithm. The comprehensive experimental evaluation results demonstrate that the estimated quantiles of the proposed algorithm are highly accurate than existing methods in the literature on both synthetic and real-world datasets, especially on important extreme quantiles.
Zheng Lin 0003, Jun Liu 0014
PIMRC2
2017 Image retrieval by dense caption reasoning
abstract
Humans tend to understand image scene by recognizing visual elements, then conjecturing and inferring based on them, hence are able to search relevant images. In this paper, we concern about the problem of complex image retrieval by reasoning image dense captions, which is similar to the way of human perception for searching images. Specifically, we transform the problem of complex image retrieval into a dense captioning and scene graph matching issue by using structured language descriptions for retrieval. Experimental results on a novel proposed large-scale content-based image retrieval dataset demonstrate the rationality and effectiveness of our method.
Xinru Wei, Yonggang Qi, Jun Liu 0014, Fang Liu 0026
VCIP3
2016 Sketch-based image retrieval via Siamese convolutional neural network
abstract
Sketch-based image retrieval (SBIR) is a challenging task due to the ambiguity inherent in sketches when compared with photos. In this paper, we propose a novel convolutional neural network based on Siamese network for SBIR. The main idea is to pull output feature vectors closer for input sketch-image pairs that are labeled as similar, and push them away if irrelevant. This is achieved by jointly tuning two convolutional neural networks which linked by one loss function. Experimental results on Flickr15K demonstrate that the proposed method offers a better performance when compared with several state-of-the-art approaches.
Yonggang Qi, Yi-Zhe Song, Honggang Zhang 0002, Jun Liu 0014
ICIP4
2016 Request Dependency Graph: A Model for Web Usage Mining in Large-Scale Web of Things
abstract
In the Web of Things (WoT) environment, Web traffic logs contain valuable information of how people interact with smart devices and Web servers. Mining the wealth of information available in the Web access logs has theoretical and practical significance for many important applications like network optimization and security management. The first critical step of the mining task is modeling the relationships among HyperText Transfer Protocol (HTTP) requests for accessing Web objects to investigate the behavior of Web clients. In this paper, we introduce the request dependency graph (RDG), a graph representation of the relationships among HTTP requests. Conceptually, a directed link from A to B in the graph means that the accessing of Web object B is caused by the accessing of A, i.e., B depends on A. We propose a methodology to establish such a graph by mining the temporal and causal information among aggregated HTTP requests. To demonstrate the value and effectiveness of the proposed model, we design and implement an algorithm for primary requests identification, which is a critical task of Web usage mining, based on the RDG. Evaluation results from a large-scale real-world Web access log shows that the RDG is a useful tool for Web usage mining.
Jun Liu 0014, Nirwan Ansari
IEEE Internet Things J.1
2015 SONAR: A scalable stream-oriented system for real-time network traffic measurements
abstract
Accurate and real-time network measurements are becoming increasingly critical for a large variety of management tasks like accounting, bandwidth provisioning and security analysis. However, existing network measurement techniques have major limitations in supporting scalable and real-time traffic data monitoring and analyzing on high-speed (10Gbps and beyond) network links. Therefore, we propose a novel real-time network measurement system, named SONAR, which facilitates the convergence of real-time network monitoring and traffic analysis. We illustrate how the proposed system is designed and implemented based on streaming computing technologies, and demonstrate its capabilities with a built-in abnormal traffic detection application. The proposed system, based on real-world actualization and evaluation, has been demonstrated to be a high-performance and scalable solution for real-time network traffic measurements.
Jun Liu 0014, Yutan Du, Jie Yang 0023, Nirwan Ansari
HPSR1
2015 A diffusion mechanism over interest-based communities in Mobile Internet
Yufei Di, Yuanyuan Qiao 0002, Jie Yang 0023, Jun Liu 0014
QSHINE4
2014 Identifying website communities in mobile internet based on affinity measurement
Jun Liu 0014, Nirwan Ansari
Comput. Commun.1