Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Marcello Maggioni

dblp:292/4246 · DBLP profile ↗
← Back
1ranked-venue papers
0as first author
1since 2021 · last 2023
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Systems, architecture and hardware · 1 · 1 since 2021Software engineering, systems software and programming languages · 1 · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Efficient and distributed learning · 87% Language models and text generation · 13%
Computer architecture, parallel and distributed computing, and storage systems
1 paper
Parallel and multicore computing · 50% Distributed systems · 50%

Topics — the 5 heaviest of 5, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Efficient and distributed learning
distributed training
0.712023
Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models · ASPLOS (1) 2023
Machine learning › Efficient and distributed learning › distributed training
model parallelism
0.712023
Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models · ASPLOS (1) 2023
Distributed systems › communication optimization
communication-computation overlap
0.712023
Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models · ASPLOS (1) 2023
Parallel and multicore computing
parallel programming models
0.712023
Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models · ASPLOS (1) 2023
Natural language and speech › Language models and text generation
large language model
0.212023
Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models · ASPLOS (1) 2023

Methods — techniques the papers use, named apart from their topics

intra-layer model parallelism · 1.3computation decomposition · 1.3
YearPublicationVenuePosition
2023 Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models
abstract
Large deep learning models have shown great potential with state-of-the-art results in many tasks. However, running these large models is quite challenging on an accelerator (GPU or TPU) because the on-device memory is too limited for the size of these models. Intra-layer model parallelism is an approach to address the issues by partitioning individual layers or operators across multiple devices in a distributed accelerator cluster. But, the data communications generated by intra-layer model parallelism can contribute to a significant proportion of the overall execution time and severely hurt the computational efficiency.
Jinliang Wei, Amit Sabne, Andy Davis, Berkin Ilbeyi, Blake Hechtman, Dehao Chen, Karthik Srinivasa Murthy, Marcello Maggioni, Tongfei Guo, Yuanzhong Xu, Zongwei Zhou
ASPLOS (1)9