Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Tomomasa Hara

dblp:390/3258 · DBLP profile ↗
← Back
1ranked-venue papers
1as first author
1since 2021 · last 2026
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Representation and self-supervised learning · 88% Language models and text generation · 12%

Topics — the 4 heaviest of 4, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Machine learning › Representation and self-supervised learning
second-order statistics
1.012026
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings · ACL (1) 2026
Machine learning › Representation and self-supervised learning
text embedding
1.012026
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings · ACL (1) 2026
Machine learning › Representation and self-supervised learning
contrastive learning
0.312026
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings · ACL (1) 2026
Natural language and speech › Language models and text generation › text representation
text encoder
0.312026
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings · ACL (1) 2026

Methods — techniques the papers use, named apart from their topics

mean pooling · 1.0contrastive fine-tuning · 1.0
YearPublicationVenuePosition
2026 Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings
abstract
For constructing text embeddings, mean pooling, which averages token embeddings, is the standard approach.This paper examines whether mean pooling actually works well in real text encoders.First, we note that mean pooling can collapse information beyond the first-order statistics of the token embeddings, such as second-order statistics that capture their spatial structure, potentially mapping distinct token embedding distributions to similar text embeddings.Motivated by this concern, we propose a simple metric to quantify such a collapse induced by mean pooling.Then, using this metric, we empirically measure how often this collapse arises in actual models and texts, and find that mean pooling works well in modern text encoders.In particular, this collapse is less likely to arise in contrastive fine-tuned text encoders than in their pretrained backbone models.We also find that the robustness of these text encoders to collapse stems from the concentration of token embeddings within each text.In addition, we find that robustness to this collapse, as quantified by our proposed metric, correlates with downstream task performance.Overall, our findings help explain why modern text encoders remain effective despite relying on seemingly coarse mean pooling.
Tomomasa Hara, Hiroto Kurita, Masaaki Imaizumi, Kentaro Inui, Sho Yokoi
ACL (1)1