Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Zheng-An Chen

dblp:371/4125 · DBLP profile ↗
← Back
1ranked-venue papers
1as first author
1since 2021 · last 2025
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Deep learning architectures and training · 61% Learning theory · 39%

Topics — the 4 heaviest of 4, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Learning theory › neural network theory › neural network analysis
rank collapse
0.912025
From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025
Machine learning › Deep learning architectures and training
training dynamics
0.912025
From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025
Machine learning › Deep learning architectures and training › training dynamics
transformer training dynamics
0.912025
From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025
Machine learning › Learning theory › learning dynamics
gradient flow analysis
0.312025
From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025

Methods — techniques the papers use, named apart from their topics

linearized analysis · 0.9gradient flow · 0.9directional convergence · 0.9
YearPublicationVenuePosition
2025 From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
abstract
Although transformer-based models have shown exceptional empirical performance, the fundamental principles governing their training dynamics are inadequately characterized beyond configuration-specific studies. Inspired by empirical evidence showing improved reasoning capabilities under small initialization scales in language models, we employ the gradient flow analytical framework established in \cite{zhou2022towards} to systematically investigate linearized Transformer training dynamics. Our theoretical analysis dissects the dynamics of attention modules into two distinct stages. In the first stage, asymmetric weight perturbations from random initialization sustain non-degenerate gradient dynamics in parameter matrices, facilitating systematic escape from small initialization regimes. Subsequently, these matrices undergo condensation, progressively aligning toward the target orientation. In the second stage, the previously static key-query matrices actively participate in training, driving the normalized matrices toward asymptotic rank collapse. This two-stage framework generalizes classical directional convergence results.
Zheng-An Chen, Tao Luo 0012
NeurIPS1