Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Dongyang Zhang 0001

dblp:156/7086-1 · DBLP profile ↗
← Back
41ranked-venue papers
9as first author
32since 2021 · last 2026
0000-0002-4839-0234ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 19 · 5 first-author · 14 since 2021Artificial intelligence and machine learning · 17 · 3 first-author · 13 since 2021Databases, data management, data science and information retrieval · 7 · 7 since 2021Applied, interdisciplinary, general and emerging computing · 5 · 1 first-author · 5 since 2021Systems, architecture and hardware · 1 · 1 since 2021Computer networks · 1 · 1 first-author

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
9 papers
Efficient and distributed learning · 34% Generative modeling · 18% Deep learning architectures and training · 13%
Databases, data mining, and information retrieval
4 papers
Knowledge graphs · 38% Recommender systems · 31% Information retrieval · 31%
Network and information security
1 paper
Security and privacy of machine learning · 100%
Computer graphics and multimedia
2 papers
Image and video processing · 100%

Topics — the 30 heaviest of 32, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Generative modeling
diffusion model
1.622025
Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition · CVPR 2025
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation · ACM Multimedia 2024
Knowledge graphs
knowledge graph embedding
1.222023
DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion · IEEE Trans. Knowl. Data Eng. 2023
DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion: (Extended Abstract) · ICDE 2022
Knowledge graphs
link prediction
1.222023
DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion · IEEE Trans. Knowl. Data Eng. 2023
DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion: (Extended Abstract) · ICDE 2022
Machine learning › Graph learning
graph reasoning
1.012026
GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understanding · WWW 2026
Natural language and speech › Language models and text generation
large language model reasoning
1.012026
GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understanding · WWW 2026
Knowledge, reasoning and agents › Multi-agent systems
multi-agent collaboration
1.012026
GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understanding · WWW 2026
Information retrieval › retrieval-augmented generation
graph-based retrieval-augmented generation
1.012026
KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation · WWW 2026
Information retrieval
retrieval-augmented generation
1.012026
KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation · WWW 2026
Recommender systems
session-based recommendation
1.012026
Bid Farewell to Seesaw: Towards Accurate Long-Tail Session-Based Recommendation via Dual Constraints of Hybrid Intents · AAAI 2026
Recommender systems › user modeling
user intent modeling
1.012026
Bid Farewell to Seesaw: Towards Accurate Long-Tail Session-Based Recommendation via Dual Constraints of Hybrid Intents · AAAI 2026
Image and video processing › super-resolution
image super-resolution
1.022021
Large Factor Image Super-Resolution With Cascaded Convolutional Neural Networks · IEEE Trans. Multim. 2021
PFFN: Progressive Feature Fusion Network for Lightweight Image Super-Resolution · ACM Multimedia 2021
Security and privacy of machine learning › retrieval-augmented generation security
knowledge poisoning
1.012026
KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation · WWW 2026
Security and privacy of machine learning
poisoning attack
1.012026
KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation · WWW 2026
Machine learning › Deep learning architectures and training › backpropagation
backpropagation through time
0.912025
Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation · NeurIPS 2025
Machine learning › Efficient and distributed learning › data selection
coreset selection
0.912025
Adaptive Dataset Quantization · AAAI 2025
Machine learning › Efficient and distributed learning › data reduction
dataset compression
0.912025
Adaptive Dataset Quantization · AAAI 2025
Machine learning › Efficient and distributed learning
dataset distillation
0.912025
Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation · NeurIPS 2025
Machine learning › Efficient and distributed learning › data reduction
dataset quantization
0.912025
Adaptive Dataset Quantization · AAAI 2025
Machine learning › Generative modeling › diffusion model
diffusion-based perception
0.912025
Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition · CVPR 2025
Machine learning › Deep learning architectures and training
training optimization
0.912025
Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation · NeurIPS 2025
Machine learning › Transfer learning and domain adaptation › domain adaptation
unsupervised domain adaptation
0.912025
Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition · CVPR 2025
Computer vision › Vision and language
vision-language model
0.912025
Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition · CVPR 2025
Machine learning › Efficient and distributed learning › model compression › knowledge distillation
data-free knowledge distillation
0.812024
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation · ACM Multimedia 2024
Machine learning › Generative modeling › diffusion model
diffusion-based data augmentation
0.812024
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation · ACM Multimedia 2024
Computer vision › Image recognition and object detection
human-object interaction detection
0.812024
Towards Open-vocabulary HOI Detection with Calibrated Vision-language Models and Locality-aware Queries · ACM Multimedia 2024
Machine learning › Efficient and distributed learning › model compression
knowledge distillation
0.812024
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation · ACM Multimedia 2024
Machine learning › Efficient and distributed learning
model compression
0.812024
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation · ACM Multimedia 2024
Machine learning › Deep learning architectures and training
convolutional neural network
0.512021
Large Factor Image Super-Resolution With Cascaded Convolutional Neural Networks · IEEE Trans. Multim. 2021
Machine learning › Efficient and distributed learning › model compression
lightweight neural network
0.512021
PFFN: Progressive Feature Fusion Network for Lightweight Image Super-Resolution · ACM Multimedia 2021
Machine learning › Graph learning › graph neural network › attention-based graph neural network
graph attention network
0.212023
DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion · IEEE Trans. Knowl. Data Eng. 2023

Methods — techniques the papers use, named apart from their topics

contrastive learning · 1.9relational graph attention network · 1.2mutual information maximization · 1.2tool calling · 1.0subgraph sampling · 1.0spectral clustering · 1.0multi-agent framework · 1.0pseudo-label generation · 0.9low-rank hessian approximation · 0.9knowledge-aligned counterfactual enhancement · 0.9diffusion guidance · 0.9dataset distillation · 0.9adaptive sampling · 0.9graph infomax · 0.6progressive attention block · 0.5dual-attention module · 0.5curriculum learning · 0.5cosine annealing · 0.5
YearPublicationVenuePosition
2026 Bid Farewell to Seesaw: Towards Accurate Long-Tail Session-Based Recommendation via Dual Constraints of Hybrid Intents
abstract
Session-based recommendation (SBR) aims to predict anonymous users' next interaction based on their interaction sessions. In practical recommendation scenario, low-exposure items constitute the majority of interactions, creating a long-tail distribution that severely compromises recommendation diversity. Existing approaches attempt to address this issue by promoting tail items but incur accuracy degradation, exhibiting a "see-saw" effect between long-tail and accuracy performance. We attribute such conflict to session-irrelevant noise within the tail item set, which existing long-tail approaches fail to identify and constrain effectively. To resolve our fundamental conflict, we propose HID (Hybrid Intent-based Dual Constraint Framework), a plug-and-play framework that transforms the conventional "see-saw" into a "win-win" relationship through introducing the hybrid intent-based dual constraints. Two key innovations are incorporated in this framework: (i) Hybrid Intent Learning, where we reformulate the intent extraction strategies by employing attribute-aware spectral clustering to reconstruct the item-to-intent mapping. Furthermore, discrimination of session-irrelevant noise is achieved through the assignment of both target and noise intents to each sessions. (ii) Intent Constraint Loss, where we propose two novel constraint paradigms regarding the diversity and accuracy to regulate the representation learning process, and unify the two optimization objectives into a unique loss. Extensive experiments across multiple SBR models and datasets demonstrate that HID can enhance both long-tail performance and recommendation accuracy, establishing new state-of-the-art performance in long-tail recommender systems.
Xiao Wang 0055, Ke Qin, Dongyang Zhang 0001, Xiurui Xie, Shuang Liang 0002
AAAI3
2026 KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation
Qizhi Chen 0001, Muquan Li, Rongzheng Wang, Dongyang Zhang 0001, Ke Qin, Shuang Liang 0002
WWW6
2026 GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understanding
abstract
Large language models (LLMs) show promising performance on small-scale graph reasoning tasks but fail when handling real-world graphs with complex queries. This phenomenon arises from LLMs' working memory constraints, which result in their inability to retain long-range graph topology over extended contexts while sustaining coherent multi-step reasoning. However, real-world graphs are often structurally complex, such as Web, Transportation, Social, and Citation networks. To address these limitations, we propose GraphCogent, a collaborative agent framework inspired by human Working Memory Model that decomposes graph reasoning into specialized cognitive processes: sense, buffer, and execute. The framework consists of three modules: Sensory Module standardizes diverse graph text representations via subgraph sampling, Buffer Module integrates and indexes graph data across multiple formats, and Execution Module combines tool calling and tool creation for efficient reasoning. We also introduce Graph4real, a comprehensive benchmark that contains four domains of real-world graphs (Web, Transportation, Social, and Citation) to evaluate LLMs' graph reasoning capabilities. Our Graph4real covers 21 different graph reasoning tasks, categorized into three types (Structural Querying, Algorithmic Reasoning, and Predictive Modeling tasks), with graph scales up to 10 times larger than existing benchmarks. Experiments show that Llama3.1-8B based GraphCogent achieves a 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B). Compared to state-of-the-art code-based baseline, our framework outperforms by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks.
Rongzheng Wang, Shuang Liang 0002, Qizhi Chen 0001, Muquan Li, Yizhuo Ma, Dongyang Zhang 0001, Ke Qin, Man-Fai Leung
WWW7
2026 SEMQ: Efficient non-uniform quantization with sensitivity-based error minimization for large language models
Dongmin Li 0001, Xiurui Xie, Dongyang Zhang 0001, Athanasios V. Vasilakos, Man-Fai Leung
Future Gener. Comput. Syst.3
2026 Modular knowledge distillation for MIM pre-trained models
Dongyang Zhang 0001, Guiduo Duan
Neurocomputing4
2026 Rethinking attention cues: Multi-Factor guided token pruning for efficient vision-language understanding
Deng Luo, Dongyang Zhang 0001, Qiuhao Xie, Cencen Liu, Qiang Dong, Xiurui Xie
Knowl. Based Syst.2
2026 Lifelong scene graph generation
Tao He 0007, Tongtong Wu, Dongyang Zhang 0001, Ming Li 0073, Yuan-Fang Li, F. Richard Yu
Pattern Recognit.4
2026 Efficient Dataset Distillation via Generative Pruning
abstract
Dataset distillation (DD) has demonstrated the promise of synthesizing smaller datasets that enable competitive performance. While most of DD methods operate in pixel space, they often suffer from poor scalability on high-resolution datasets. Recent works have thus shifted towards parameterizing synthetic data using deep generative priors. However, these approaches apply latent updates to all generator layers, leading to substantial computational overhead. To address this limitation, we propose Generative Lightweight Distillation (GLiD), a unified framework that jointly compresses the generator and accelerates latent optimization for efficient distillation. GLiD introduces two key components: (1) We introduce a Classification-Diversity Sensitivity Pruning mechanism that quantifies both discriminative utility and semantic diversity of each output channel to guide structural pruning and selective layer-wise optimization. (2) We also present a Layer-Adaptive Scheduling strategy that dynamically allocates latent update steps across generator stages based on convergence behavior. Extensive experiments on CIFAR-10 and ImageNet-1K and its subsets demonstrate that our GLiD achieves up to 10× acceleration across different datasets, while maintaining performance competitive with state-of-the-art methods.
Yingyi Ma, Muquan Li, Guiduo Duan, Ke Qin, Shuang Liang 0002, Dongyang Zhang 0001
IEEE Trans. Big Data6
2026 NDM: Boosting Dataset Distillation via Nested Difficulty Matching
Dongyang Zhang 0001, Hang Gou, Yue Zhang 0042, Dan Song 0006, Xiurui Xie
IEEE Trans. Circuits Syst. Video Technol.1
2026 Efficient Industrial Dataset Distillation With Textual Trajectory Matching
abstract
Modern industrial environments generate massive streams of discrete textual logs that record device status, error codes, and operational events. Deploying models on resource-constrained edge devices demands extreme data compression and fast inference, yet existing dataset distillation (DD) methods are designed for images and capture only short-term training dynamics via single-step gradient or distribution matching. To address these limitations, we propose textual trajectory matching (TTM), a novel textual DD framework that aligns student trajectories with expert trajectories derived from industrial data training. We introduce a Mask-and-Fill initialization to enhance trajectory diversity, expanding semantic representations. We further propose a manifold distribution-based initialization to preserve original semantic features through low-dimensional manifold analysis. To address model computational costs, we design a subset matching strategy to reducing by aligning critical structural components. Evaluated on SST-2, MNLI-m, AGNews, and BlueGene/L (BGL) with$\text{BERT}_{\text{BASE}}$,$\text{RoBERTa}_{\text{BASE}}$,$\text{XLNet}_{\text{BASE}}$, and LLaMA 3, TTM achieves 3.6% higher accuracy than state-of-the-art methods, enabling efficient industrial dataset and model deployment in smart factories.
Muquan Li, Dongyang Zhang 0001, Ke Qin, Guangchun Luo
IEEE Trans. Ind. Informatics3
2026 From Tradeoff to Synergy: Rethinking the Long-tail Session-based Recommendation via Dual Constraints of Hybrid Intent
abstract
Session-based recommendation (SBR) predicts the next interaction of anonymous users based on their session data. In practical scenarios, low-exposure tail items constitute the majority of interactions, leading to the biased recommendation whereby a small set of high-exposure head items is preferentially recommended. Existing solutions focus on promoting those tail items but with a sacrifice in recommendation accuracy. We attribute such a tradeoff to session-irrelevant noise within the tail items, which existing long-tail approaches fail to identify and constrain effectively. To resolve this fundamental conflict, we propose Hybrid Intent-based Dual Constraint framework (HID) , a plug-and-play framework that converts the traditional tradeoff between long-tail performance and accuracy into a synergistic relationship by introducing hybrid intent-based dual constraints. HID incorporates two key innovations: (i) Hybrid Intent Learning , which reformulates intent extraction through attribute-aware spectral clustering and discriminates session-irrelevant noise by assigning target and noise intents to each session and (ii) Intent Constraint Loss , which introduces two constraints—diversity and accuracy—to regulate item and session representation learning. These two objectives are unified into a single training loss through rigorous theoretical derivation. To handle ambiguous user intent in the real-world, we also propose a slack version of HID. Extensive experiments show that HID enhances both long-tail performance and accuracy, establishing new state-of-the-art performance in long-tail recommender systems. The implementation code is available at: https://github.com/jarviswww/Code4TradeSynergy .
Xiao Wang 0055, Ke Qin, Dongyang Zhang 0001, Xiurui Xie, Shuang Liang 0002
ACM Trans. Inf. Syst.3
2025 Adaptive Dataset Quantization
abstract
Contemporary deep learning, characterized by the training of cumbersome neural networks on massive datasets, confronts substantial computational hurdles. To alleviate heavy data storage burdens on limited hardware resources, numerous dataset compression methods such as dataset distillation (DD) and coreset selection have emerged to obtain a compact but informative dataset through synthesis or selection for efficient training. However, DD involves an expensive optimization procedure and exhibits limited generalization across unseen architectures, while coreset selection is limited by its low data keep ratio and reliance on heuristics, hindering its practicality and feasibility. To address these limitations, we introduce a newly versatile framework for dataset compression, namely Adaptive Dataset Quantization (ADQ). Specifically, we first identify the sub-optimal performance of naive Dataset Quantization (DQ), which relies on uniform sampling and overlooks the varying importance of each generated bin. Subsequently, we propose a novel adaptive sampling strategy through the evaluation of generated bins' representativeness score, diversity score and importance score, where the former two scores are quantified by the texture level and contrastive learning-based techniques, respectively. Extensive experiments demonstrate that our method not only exhibits superior generalization capability across different architectures, but also attains state-of-the-art results.
Muquan Li, Dongyang Zhang 0001, Qiang Dong, Xiurui Xie, Ke Qin
AAAI2
2025 Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition
abstract
Visual Emotion Recognition (VER) is a critical yet challenging task aimed at inferring emotional states of individuals based on visual cues. However, existing works focus on single domains, e.g., realistic images or stickers, limiting VER models’ cross-domain generalizability. To fill this gap, we introduce an Unsupervised Cross-Domain Visual Emotion Recognition (UCDVER) task, which aims to generalize visual emotion recognition from the source domain (e.g., realistic images) to the low-resource target domain (e.g., stickers) in an unsupervised manner. Compared to the conventional unsupervised domain adaptation problems, UCDVER presents two key challenges: a significant emotional expression variability and an affective distribution shift. To mitigate these issues, we propose the Knowledge-aligned Counterfactual-enhancement Diffusion Perception (KCDP) framework. Specifically, KCDP leverages a VLM to align emotional representations in a shared knowledge space and guides diffusion models for improved visual affective perception. Furthermore, a Counterfactual-Enhanced Language-image Emotional Alignment (CLIEA) method generates high-quality pseudo-labels for the target domain. Extensive experiments demonstrate that our model surpasses SOTA models in both perceptibility and generalization, e.g., gaining 12% improvements over SOTA VER model TGCA-PVT. The project page is at https://yinwen2019.github.io/ucdver/.
Guiduo Duan, Dongyang Zhang 0001, Yuan-Fang Li, Tao He 0007
CVPR4
2025 Fine-grained Block Pruning with Tiny Sets for Vision Transformers
abstract
Vision Transformers (ViTs) and their variants have achieved remarkable success across a broad spectrum of computer vision tasks. However, their high computational cost and significant data requirements present challenges for deployment in resource-constrained environments. Current pruning methods for ViTs predominantly focus on reducing token counts, which often disrupt the inherent spatial structure of ViTs, hindering their adaptability to hardware. Furthermore, how to compress ViTs efficiently in few-shot scenarios remains an open question. Hence, we introduce a fine-grained block pruning framework for ViTs, named FBP-ViT. Unlike traditional block pruning techniques that indiscriminately remove entire blocks, FBP-ViT selectively eliminates Multi-Head Self-Attention (MSA) or Multi-Layer Perceptron (MLP) blocks, offering enhanced flexibility and efficiency. We unify pruning and finetuning, ensuring practicality in resource-constrained environments for real-world applications. We evaluate the proposed FBP-ViT across ViTs of varying sizes and architectures, demonstrating its effectiveness in improving computational efficiency while maintaining high performance. Specifically, with a speedup factor of 1.34, FBP-ViT preserves 80.39% top-1 accuracy on ImageNet-1k using DeiT-Base, achieving more precise and efficient pruning with tiny sets.
Yilin Wang 0028, Qiang Dong, Dongyang Zhang 0001, Tao He 0007
ICMR3
2025 Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation
abstract
The growing demand for efficient deep learning has positioned dataset distillation as a pivotal technique for compressing training dataset while preserving model performance. However, existing inner-loop optimization methods for dataset distillation typically rely on random truncation strategies, which lack flexibility and often yield suboptimal results. In this work, we observe that neural networks exhibit distinct learning dynamics across different training stages—early, middle, and late—making random truncation ineffective. To address this limitation, we propose Automatic Truncated Backpropagation Through Time (AT-BPTT), a novel framework that dynamically adapts both truncation positions and window sizes according to intrinsic gradient behavior. AT-BPTT introduces three key components: (1) a probabilistic mechanism for stage-aware timestep selection, (2) an adaptive window sizing strategy based on gradient variation, and (3) a low-rank Hessian approximation to reduce computational overhead. Extensive experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K show that AT-BPTT achieves state-of-the-art performance, improving accuracy by an average of 6.16\% over baseline methods. Moreover, our approach accelerates inner-loop optimization by 3.9 × while saving 63\% memory cost.
Muquan Li, Hang Gou, Dongyang Zhang 0001, Shuang Liang 0002, Xiurui Xie, Deqiang Ouyang, Ke Qin
NeurIPS3
2025 SRMamba-T: Exploring the hybrid Mamba-Transformer network for Single Image Super-Resolution
Cencen Liu, Dongyang Zhang 0001, Guoming Lu, Jielei Wang, Guangchun Luo
Neurocomputing2
2025 Toward lightweight image super-resolution via re-parameterized kernel recalibration
Dongyang Zhang 0001, Jiachi Liu, Shuang Liang 0002, Xiurui Xie, Qiang Dong, Ke Qin
Knowl. Based Syst.1
2025 DDM4TST: Diffusion Model for Fine-grained Text Style Transfer by Disentangled Representation
abstract
Fine-grained Text Style Transfer (FTST) aims to make targeted and precise modifications to specific stylistic components of a sentence. Existing methods typically attempt to disentangle style and content representations for FTST. However, style and content are inherently abstract, making them difficult to formalize and manipulate independently. To address this, we propose a novel framework, Disentanglement Diffusion Model for Text Style Transfer (DDM4TST) , which reformulates style transformation as either semantic or syntactic transformation. By learning disentangled representations, the model enables accurate and fine-grained style control. Specifically, we construct a feature parsing module to effectively separate semantic and syntactic representations. We then incorporate a diffusion model conditioned on these disentangled representations, allowing for fine-grained control of stylistic attributes while preserving the core content of the original sentence. This integration into the denoising process enhances the controllability and precision of the style transformation. Extensive experiments on the benchmark StylePTB dataset demonstrate that our model consistently outperforms widely adopted baselines. The results validate the effectiveness of our approach in achieving high-quality style transformation while maintaining content fidelity.
Cencen Liu, Qiugang Zhan, Dongyang Zhang 0001, Raza Ahmad
ACM Trans. Asian Low Resour. Lang. Inf. Process.5
2024 Knowledge Distillation for Single Image Super-Resolution via Contrastive Learning
abstract
In recent years, thanks to the vigorous development of deep learning, single image super-resolution has advanced greatly. Most super-resolution (SR) methods use convolution layers to construct the network, which achieves superior results over the traditional methods based on manual features. However, most methods based on convolutional neural networks (CNN) blindly deepen the depth of the network leading to a large number of model parameters, which inevitably brings huge computing overhead and memory consumption, and greatly limits the application in resource-limited devices. In order to alleviate this problem, a knowledge distillation framework based on contrastive learning is proposed to compress and accelerate the SR model with enormous parameters. The student network is directly constructed by reducing the number of layers of the teacher network. In particular, the proposed method distills the statistical information of the intermediate feature maps from the teacher network to train the lightweight student network. In addition, through explicit knowledge transfer, a novel contrastive loss is introduced to improve the reconstruction performance of the student network. Experiments show that the proposed contrastive distillation framework can effectively compress the model scale with an acceptable loss of performance.
Cencen Liu, Dongyang Zhang 0001, Ke Qin
ICMR2
2024 Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation
abstract
Data-free knowledge distillation (DFKD) has emerged as a pivotal technique in the domain of model compression, substantially reducing the dependency on the original training data. Nonetheless, conventional DFKD methods that employ synthesized training data are prone to the limitations of inadequate diversity and discrepancies in distribution between the synthesized and original datasets. To address these challenges, this paper introduces an innovative approach to DFKD through diverse diffusion augmentation (DDA). Specifically, we revise the paradigm of common data synthesis in DFKD to a composite process through leveraging diffusion models subsequent to data synthesis for self-supervised augmentation, which generates a spectrum of data samples with similar distributions while retaining controlled variations. Furthermore, to mitigate excessive deviation in the embedding space, we introduce an image filtering technique grounded in cosine similarity to maintain fidelity during the knowledge distillation process. Comprehensive experiments conducted on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets showcase the superior performance of our method across various teacher-student network configurations, outperforming the contemporary state-of-the-art DFKD methods. Code will be available at: https://github.com/SLGSP/DDA.
Muquan Li, Dongyang Zhang 0001, Tao He 0007, Xiurui Xie, Yuan-Fang Li, Ke Qin
ACM Multimedia2
2024 Towards Open-vocabulary HOI Detection with Calibrated Vision-language Models and Locality-aware Queries
abstract
The open-vocabulary human-object interaction (Ov-HOI) detection aims to identify both base and novel categories of human-object interactions while only base categories are available during training. Existing Ov-HOI methods commonly leverage knowledge distilled from CLIP to extend their ability to detect previously unseen interaction categories. However, our empirical observations indicate that the inherent noise present in CLIP has a detrimental effect on HOI prediction. Moreover, the absence of novel human-object position distributions often leads to overfitting on the base categories within their learned queries. To address these issues, we propose a two-step framework named, CaM-LQ, Calibrating visual-language Models, (e.g., CLIP) for open-vocabulary HOI detection with Locality-aware Queries. By injecting the fine-grained HOI supervision from the calibrated CLIP into the HOI decoder, our model can achieve the goal of predicting novel interactions. Extensive experimental results demonstrate that our approach performs well in open-vocabulary human-object interaction detection, surpassing state-of-the-art methods across multiple metrics on mainstream datasets and showing superior open-vocabulary HOI detection performance, e.g., with 4.54 points improvement on the HICO-DET dataset over the SoTA CLIP4HOI on the UV task with the same backbone ResNet-50.
Zhenhao Yang, Deqiang Ouyang, Guiduo Duan, Dongyang Zhang 0001, Tao He 0007, Yuan-Fang Li
ACM Multimedia5
2024 Towards Elastic Image Super-Resolution Network via Progressive Self-distillation
Xin'an Yu, Dongyang Zhang 0001, Cencen Liu, Qiang Dong, Guiduo Duan
PRCV (8)2
2023 Capturing Cross-Scale Disparity for Stereo Image Super-Resolution
abstract
Stereo image super-resolution (SR) exploits the stereo feature information from cross-view image pairs for image resolution. This paper focuses on how to effectively exploit the disparity information between stereo viewpoints and proposes a cross-scale parallax-attention network (CSPAN) for stereo image SR. Specifically, a novel cross-scale parallax attention module (CPAM) is developed to explore cross-scale parallax prior. Moreover, instead of the widely used upsampling module based on the sub-pixel layer, we present a novel cascade dynamic upsampling module (CDUM), which not only dynamically generates the upsampling filters according to the input content, but also restores the high frequency details in a coarse-to-fine manner. Especially, nonlinear activation free blocks (NAFBlocks) are used as the feature extraction module in our network, which further improves the performance of our model. Extensive experiments on Mid- dlebury, KITTI 2012 and KITTI 2015 demonstrate that the proposed framework outperforms many competitive stereo SR methods in both PSNR and SSIM. Code is available at https://github.com/DoragonKuesuto/CSPAN.
Changyu Li, Dongyang Zhang 0001, Jie Shao 0001
ICASSP3
2023 DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion
abstract
Recently, many knowledge graph embedding models for knowledge graph completion have been proposed, ranging from the initial translation-based models such as TransE to recent convolutional neural network (CNN) models such as ConvE. However, these models only focus on semantic information of knowledge graph and neglect the natural graph structure information. Although graph convolutional network (GCN)-based models for knowledge graph embedding have been introduced to address this issue, they still suffer from fact incompleteness, resulting in the unconnectedness of knowledge graph. To solve this problem, we propose a novel model called deep relational graph infomax (DRGI) with mutual information (MI) maximization which takes the benefit of complete structure information and semantic information together. Specifically, the proposed DRGI consists of two encoders which are two identical adaptive relational graph attention networks (ARGATs), corresponding to catching semantic information and complete structure information respectively. Our method establishes new state-of-the-art on the standard datasets for knowledge graph completion. In addition, by exploring the complete structure information, DRGI embraces the merits of faster convergence speed over existing methods and better predictive performance for entities with small indegree.
Shuang Liang 0002, Jie Shao 0001, Dongyang Zhang 0001, Bin Cui 0001
IEEE Trans. Knowl. Data Eng.3
2022 Wide Feature Projection with Fast and Memory-Economic Attention for Efficient Image Super-Resolution
Minghao Fu 0002, Dongyang Zhang 0001, Changyu Li, Jie Shao 0001
BMVC2
2022 DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion: (Extended Abstract)
abstract
Recently, many knowledge graph embedding models for knowledge graph completion have been proposed, ranging from the initial translation-based models such as TransE to recent convolutional neural network (CNN) models such as ConvE. However, these models only focus on semantic information of knowledge graph and neglect the natural graph structure information. Although graph convolutional network (GCN)-based models for knowledge graph embedding have been introduced to address this issue, they still suffer from fact incompleteness, resulting in the unconnectedness of knowledge graph. To solve this problem, we propose a novel model called deep relational graph infomax (DRGI) with mutual information (MI) maximization which takes the benefit of complete structure information and semantic information together. Specifically, the proposed DRGI consists of two encoders which are two identical adaptive relational graph attention networks (ARGATs), corresponding to catching semantic information and complete structure information respectively. Our method establishes new state-of-the-art on the standard datasets for knowledge graph completion.
Shuang Liang 0002, Jie Shao 0001, Dongyang Zhang 0001, Bin Cui 0001
ICDE3
2021 Efficient Spatio-Temporal Network with Gated Fusion for Video Super-Resolution
Changyu Li, Dongyang Zhang 0001, Ning Xie 0003, Jie Shao 0001
ICANN (5)2
2021 Learning Multi-dimensional Parallax Prior for Stereo Image Super-Resolution
Changyu Li, Dongyang Zhang 0001, Chunlin Jiang, Ning Xie 0003, Jie Shao 0001
ICONIP (6)2
2021 PFFN: Progressive Feature Fusion Network for Lightweight Image Super-Resolution
abstract
Recently, convolutional neural network (CNN) has been the core ingredient of modern models, triggering the surge of deep learning in super-resolution (SR). Despite the great success of these CNN-based methods which are prone to be deeper and heavier, it is impracticable to directly apply these methods for some low-budget devices due to the superfluous computational overhead. To alleviate this problem, a novel lightweight SR network named progressive feature fusion network (PFFN) is developed to seek for better balance between performance and running efficiency. Specifically, to fully exploit the feature maps, a novel progressive attention block (PAB) is proposed as the main building block of PFFN. The proposed PAB adopts several parallel but connected paths with pixel attention, which could significantly increase the receptive field of each layer, distill useful information and finally learn more discriminative feature representations. In PAB, a powerful dual attention module (DAM) is further incorporated to provide the channel and spatial attention mechanism in fairly lightweight manner. Besides, we construct a pretty concise and effective upsampling module with the help of multi-scale pixel attention, named MPAU. All of the above modules ensure the network can benefit from attention mechanism while still being lightweight enough. Furthermore, a novel training strategy following the cosine annealing learning scheme is proposed to maximize the representation ability of the model. Comprehensive experiments show that our PFFN achieves the best performance against all existing lightweight state-of-the-art SR methods with less number of parameters and even performs comparably to computationally expensive networks.
Dongyang Zhang 0001, Changyu Li, Ning Xie 0003, Guoqing Wang 0001, Jie Shao 0001
ACM Multimedia1
2021 Multi-Branch Networks for Video Super-Resolution With Dynamic Reconstruction Strategy
abstract
Recently, the rapid development of 2-dimensional (2D) convolutional neural networks (CNNs) has driven single image super-resolution (SISR) into a new era, owing to their powerful ability in modeling spatial relation within one single image. However, few studies focus on video super-resolution (VSR) due to the key challenge that apart from the spatial relation, the temporal dependence among consecutive low-resolution (LR) frames must be taken into consideration for better reconstruction. In this article, unlike most previous methods based on optical flow for motion compensation, 3-dimensional (3D) convolution is utilized to capture the temporal relation. Firstly, in contrast to the conventional 3D convolution which is notorious for the excessively high computational burden, we propose an efficient 3D convolutional block (E3DB) through convolution factorization principle (CFP), which significantly reduces the computing load while maximally maintaining the temporal information. Then, by taking advantage of E3DB, we propose a novel multi-resolution extraction block (MREB) which aggregates the information from multiple resolutions, leading to the stronger high-resolution representation learning and better feature extraction. Besides, based on our 3-branch architecture, instead of the simple addition or concatenation, a dynamic reconstruction strategy (DRS) is proposed to adaptively fuse the optimal information of temporal dependence from each branch. It is therefore termed as dynamic multiple branch network (DMBN). Comprehensive experiments on public benchmark datasets demonstrate the superiority of our DMBN over the current state-of-the-art methods in terms of accuracy and efficiency.
Dongyang Zhang 0001, Jie Shao 0001, Zhenwen Liang, Xueliang Liu, Heng Tao Shen
IEEE Trans. Circuits Syst. Video Technol.1
2021 Remote Sensing Image Super-Resolution via Mixed High-Order Attention Network
abstract
Recently, remote sensing images have become increasingly popular in a number of tasks, such as environmental monitoring. However, the observed images from satellite sensors often suffer from low-resolution (LR), making it difficult to meet the requirements for further analysis. Super-resolution (SR) aims to increase the image resolution while providing finer spatial details, which perfectly remedies the weakness of satellite images. Therefore, in this article, we propose an innovative mixed high-order attention network (MHAN) for remote sensing SR. It comprises two components: a feature extraction network for feature extraction, and a feature refinement network with high-order attention (HOA) mechanism for detail restoration. In the feature extraction network, we replace the elementwise addition with weighted channelwise concatenation in all skip connections, which greatly facilitates the information flow. In the feature refinement network, rather than exploring the first-order statistics (spatial or channel attention), we introduce the HOA module to restore the missing details. Finally, to fully exploit hierarchical features, we introduce the frequency-aware connection to bridge the feature extraction and feature refinement networks. Experiments on two widely used remote sensing image data sets demonstrate that our MHAN not only obtains better accuracy than the state-of-the-art methods but also shows the superiority in terms of running time and GPU cost. Code is available at https://github.com/ZhangDY827/MHAN.
Dongyang Zhang 0001, Jie Shao 0001, Heng Tao Shen
IEEE Trans. Geosci. Remote. Sens.1
2021 Large Factor Image Super-Resolution With Cascaded Convolutional Neural Networks
abstract
Recently, convolutional neural networks (CNNs) have attracted considerable attention in single image super-resolution (SISR) and have enabled great performance improvements. However, most of the existing methods super-resolve input images to the desired size with an interpolation operation during the beginning stage, which brings about heavy aliasing artifacts and high computational costs. Especially for large upsampling factors (e.g., 8×), it remains a challenge to restore high-quality results for deeply degraded images. To tackle this problem, we propose a cascaded super-resolution convolutional neural network (CSRCNN), which takes a single low-resolution (LR) image as an input and reconstructs high-resolution (HR) images in a progressive way. At each cascaded level, to help converge and improve the accuracy, a novel U-net based block with backprojection is first introduced, which exploits the mutual relation between HR and LR feature spaces. A refined block following the U-net block is also used to reconstruct the realistic texture details. In addition, we naturally utilize the strategy of curriculum learning, organizing the learning process from easy (small factors) to hard (large factors). Comprehensive experiments on benchmark datasets demonstrate that the proposed network achieves superior results compared with those of other state-of-the-art methods, particularly with the 8× upsampling factor.
Dongyang Zhang 0001, Jie Shao 0001, Zhenwen Liang, Lianli Gao, Heng Tao Shen
IEEE Trans. Multim.1
2020 Fused Recurrent Network Via Channel Attention For Remote Sensing Satellite Image Super-Resolution
abstract
Remote sensing satellite images often suffer from low spatial resolution. Image super-resolution plays an important role in remote sensing image processing. However, existing methods show that increasing network depth will inevitably lead to the dramatic increase of model parameters and the over-fitting problem. Besides, most methods treat different types of information (low-frequency and high-frequency) equally. Motivated by these observations, we propose a fused recurrent network via channel attention (CA-FRN) in this paper. The basic module, recursive channel attention block (RCAB), pays enough attention to the high-frequency information and diminishes the low-frequency information adaptively through channel attention. Based on RCAB, we render our model effective by retaining and fusing hierarchical local information of both low-resolution and high-resolution, and we enhance the network performance simply by increasing the number of RCABs without adding extra parameters. We evaluate the proposed model on satellite images from different datasets, and the proposed CA-FRN is superior to the state-of-the-art methods. Code is available at https://github.com/lxy0922/CAFRN.
Dongyang Zhang 0001, Zhenwen Liang, Deqiang Ouyang, Jie Shao 0001
ICME2
2020 Joint image deblurring and super-resolution with attention dual supervised network
Dongyang Zhang 0001, Zhenwen Liang, Jie Shao 0001
Neurocomputing1
2020 Traffic sign detection and recognition based on pyramidal convolutional networks
Zhenwen Liang, Jie Shao 0001, Dongyang Zhang 0001, Lianli Gao
Neural Comput. Appl.3
2020 Kernel Attention Network for Single Image Super-Resolution
abstract
Recently, attention mechanisms have shown a developing tendency toward convolutional neural network (CNN), and some representative attention mechanisms, i.e., channel attention (CA) and spatial attention (SA) have been fully applied to single image super-resolution (SISR) tasks. However, the existing architectures directly apply these attention mechanisms to SISR without much consideration of the nature characteristic, resulting in less strong representational power. In this article, we propose a novel kernel attention module (KAM) for SISR, which enables the network to adjust its receptive field size corresponding to various scales of input by dynamically selecting the appropriate kernel. Based on this, we stack multiple kernel attention modules with group and residual connection to constitute a novel architecture for SISR, which enables our network to learn more distinguishing representations through filtering the information under different receptive fields. Thus, our network is more sensitive to multi-scale features, which enables our single network to deal with multi-scale SR task by predefining the upscaling modules. Besides, other attention mechanisms in super-resolution are also investigated and illustrated in detail in this article. Thanks to the kernel attention mechanism, the extensive benchmark evaluation shows that our method outperforms the other state-of-the-art methods.
Dongyang Zhang 0001, Jie Shao 0001, Heng Tao Shen
ACM Trans. Multim. Comput. Commun. Appl.1
2019 A Fast End-to-End Method with Style Transfer for Room Layout Estimation
abstract
In this paper, we focus on the problem of estimating the spatial layout of indoor scenes from a single RGB image. Earlier solutions to this task largely rely on hand-crafted features and vanishing point detection, and recent practices usually utilize a fully convolution network (FCN) to achieve better results. However, existing works suffer from time consumption and sufficient usage of semantic information in each pixel. To tackle these issues, based on the FCN architecture, we propose a fast end-to-end method for room layout estimation. For the first time, style transfer is introduced for data enhancement, which not only extends the amount of training dataset, but also makes the pixels in images more discriminative, especially in highly cluttered rooms. Besides, we further improve the accuracy by fusing the output from multiple networks. Extensive evaluations on the public large-scale scene understanding challenge (LSUN) dataset demonstrate that our proposed method builds a new state-of-the-art result in terms of speed and accuracy.
Jie Shao 0001, Dongyang Zhang 0001, Xuehui Wu
ICME3
2019 Jointly Solving Deblurring and Super-Resolution Problems with Dual Supervised Network
abstract
Thanks to the vigorous development of deep learning techniques, the solutions of super-resolution (SR) and deblurring have made great progress in recent years. However, in real-world scenarios, the low-quality images not only suffer from the loss of spatial information (resolution), they are often degradated by complicated non-uniform motion blur as well. As a result, traditional SR and deblurring methods are not robust enough to restore those blurred and low-resolution (LR) images to sharp and high-resolution (HR) images. Therefore, we propose a novel dual supervised network (DSN) to jointly solve the SR and deblurring problems. In the beginning we use a deblurring module to make blurry input much sharper and feed the deblurred feature to a reconstruction module, which aims to produce the final HR result from the deblurred feature. Moreover, we employ dual supervised structure to fully exploit the mutual dependencies between LR and HR patches. Experiments show that our network outperforms the state-of-the-art methods.
Zhenwen Liang, Dongyang Zhang 0001, Jie Shao 0001
ICME2
2019 Unsupervised Facial Image Synthesis Using Two-Discriminator Adversarial Autoencoder Network
abstract
Recent years have witnessed the unprecedented success in single image synthesis by the means of convolutional neural networks (CNNs). High-level synthesis of facial image such as expression translation and attribute swap is still a challenging task due to high non-linearity. Previous methods suffer from the limitations that being unable to transfer multiple face attributes simultaneously, or incapability of transferring an attribute to another by a continuously changing way. To address this problem, we propose a two-discriminator adversarial autoencoder network (TAAN). The latent-discriminator is trained to disentangle an input image from its original facial attribute, while the pixel-discriminator is trained to make the output image attach to the target facial attribute. By controlling the attribute values, we can choose which and how much a specific attribute can be perceivable in the generated image. Quantitative and qualitative evaluations are conducted on the celebA and KDEF datasets, and the comparison with the state-of-the-art methods shows the competency of our proposed TAAN.
Xuehui Wu, Jie Shao 0001, Dongyang Zhang 0001
ICME3
2019 Dual-Path Recurrent Network for Image Super-Resolution
Dongyang Zhang 0001, Jie Shao 0001
ICONIP (5)2
2017 Sharp and Real Image Super-Resolution Using Generative Adversarial Network
Dongyang Zhang 0001, Jie Shao 0001, Gang Hu 0004, Lianli Gao
ICONIP (3)1