Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page
Demo Data from VLDB 2026 / reviewers in the wild / expert

Dmitry Lepikhin

dblp:141/4524 · DBLP profile ↗
← Back
2ranked-venue papers
1as first author
2since 2021 · last 2022
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 2 · 1 first-author · 2 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
2 papers
Efficient and distributed learning · 71% Deep learning architectures and training · 29%

Topics — the 8 heaviest of 8, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Efficient and distributed learning › model compression › sparsity
activation sparsity		 0.612022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts · ICML 2022
Machine learning › Efficient and distributed learning
efficient training		 0.612022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts · ICML 2022
Machine learning › Deep learning architectures and training
mixture of experts		 0.612022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts · ICML 2022
Machine learning › Efficient and distributed learning › large-scale learning
model scaling		 0.612022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts · ICML 2022
Machine learning › Deep learning architectures and training › mixture of experts
sparse mixture-of-experts		 0.612022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts · ICML 2022
Machine learning › Efficient and distributed learning › adaptive computation
conditional computation		 0.512021
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding · ICLR 2021
Machine learning › Efficient and distributed learning
distributed training		 0.512021
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding · ICLR 2021
Machine learning › Efficient and distributed learning › large-scale learning
large-scale model training		 0.112021
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding · ICLR 2021

Methods — techniques the papers use, named apart from their topics

sparse activation · 0.6mixture of experts · 0.6conditional computation · 0.5automatic sharding · 0.5
YearPublicationVenuePosition
2022 GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
abstract
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, training these large dense models requires significant amounts of computing resources. In this paper, we propose and develop a family of language models named \glam (\textbf{G}eneralist \textbf{La}nguage \textbf{M}odel), which uses a sparsely activated mixture-of-experts architecture to scale the model capacity while also incurring substantially less training cost compared to dense variants. The largest \glam has 1.2 trillion parameters, which is approximately 7x larger than GPT-3. It consumes only 1/3 of the energy used to train GPT-3 and requires half of the computation flops for inference, while still achieving better overall fewshot performance across 29 NLP tasks.
Nan Du 0002, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, William Fedus, Maarten Bosma, Zongwei Zhou, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathy Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang 0043, Quoc V. Le, Claire Cui
 ICML5
2021 GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer
 ICLR1