Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Lu Pu

dblp:144/7134 · DBLP profile ↗
← Back
2ranked-venue papers
1as first author
1since 2021 · last 2024
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1 · 1 first-author

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Generative modeling · 50% Efficient and distributed learning · 50%

Topics — the 4 heaviest of 4, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Efficient and distributed learning › model compression › transformer compression
attention compression
0.812024
DiTFastAttn: Attention Compression for Diffusion Transformer Models · NeurIPS 2024
Machine learning › Generative modeling
diffusion model
0.812024
DiTFastAttn: Attention Compression for Diffusion Transformer Models · NeurIPS 2024
Machine learning › Generative modeling › diffusion model
diffusion transformer
0.812024
DiTFastAttn: Attention Compression for Diffusion Transformer Models · NeurIPS 2024
Machine learning › Efficient and distributed learning
model compression
0.812024
DiTFastAttn: Attention Compression for Diffusion Transformer Models · NeurIPS 2024

Methods — techniques the papers use, named apart from their topics

window attention · 0.8attention sharing · 0.8
YearPublicationVenuePosition
2024 DiTFastAttn: Attention Compression for Diffusion Transformer Models
abstract
Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to the quadratic complexity of self-attention operators. We propose DiTFastAttn, a post-training compression method to alleviate the computational bottleneck of DiT. We identify three key redundancies in the attention computation during DiT inference: (1) spatial redundancy, where many attention heads focus on local information; (2) temporal redundancy, with high similarity between the attention outputs of neighboring steps; (3) conditional redundancy, where conditional and unconditional inferences exhibit significant similarity. We propose three techniques to reduce these redundancies: (1) $\textit{Window Attention with Residual Sharing}$ to reduce spatial redundancy; (2) $\textit{Attention Sharing across Timesteps}$ to exploit the similarity between steps; (3) $\textit{Attention Sharing across CFG}$ to skip redundant computations during conditional generation.
Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang 0001, Tianchen Zhao, Shengen Yan, Guohao Dai 0001, Yu Wang 0002
NeurIPS3
2019 Using "Big Data" to Determine Prevalence of Side Effects of Chemotherapy in Breast Cancer Patients
Lu Pu, David C. Kaelber
AMIA1