Demo Data from EDBT 2026 / reviewers in the wild / expert
Zheng-An Chen
dblp:371/4125
· DBLP profile ↗
1ranked-venue papers
1as first author
1since 2021 · last 2025
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Deep learning architectures and training · 61% Learning theory · 39% |
Topics — the 4 heaviest of 4, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Machine learning › Learning theory › neural network theory › neural network analysis
rank collapse |
0.9 | 1 | 2025 | From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025 |
Machine learning › Deep learning architectures and training
training dynamics |
0.9 | 1 | 2025 | From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025 |
Machine learning › Deep learning architectures and training › training dynamics
transformer training dynamics |
0.9 | 1 | 2025 | From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025 |
Machine learning › Learning theory › learning dynamics
gradient flow analysis |
0.3 | 1 | 2025 | From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics · NeurIPS 2025 |
Methods — techniques the papers use, named apart from their topics
linearized analysis · 0.9gradient flow · 0.9directional convergence · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training DynamicsabstractAlthough transformer-based models have shown exceptional empirical performance, the fundamental principles governing their training dynamics are inadequately characterized beyond configuration-specific studies. Inspired by empirical evidence showing improved reasoning capabilities under small initialization scales in language models, we employ the gradient flow analytical framework established in \cite{zhou2022towards} to systematically investigate linearized Transformer training dynamics. Our theoretical analysis dissects the dynamics of attention modules into two distinct stages. In the first stage, asymmetric weight perturbations from random initialization sustain non-degenerate gradient dynamics in parameter matrices, facilitating systematic escape from small initialization regimes. Subsequently, these matrices undergo condensation, progressively aligning toward the target orientation. In the second stage, the previously static key-query matrices actively participate in training, driving the normalized matrices toward asymptotic rank collapse. This two-stage framework generalizes classical directional convergence results. Zheng-An Chen, Tao Luo 0012 |
NeurIPS | 1 |