Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data.

Jingjing Zhang 0007

dblp:90/4994-7 · DBLP profile ↗
← Back
1ranked-venue papers
1as first author
1since 2021 · last 2022
0000-0001-9308-0000ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 first-author · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Vision and language · 93% Information extraction and text analysis · 7%

Topics — the 5 heaviest of 5, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Computer vision › Vision and language
image captioning
0.612022
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning · ACM Multimedia 2022
Computer vision › Vision and language › vision-language model
multimodal large language model
0.612022
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning · ACM Multimedia 2022
Computer vision › Vision and language › image captioning
news image captioning
0.612022
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning · ACM Multimedia 2022
Computer vision › Vision and language › vision-language model › prompt learning
vision-language model prompting
0.612022
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning · ACM Multimedia 2022
Natural language and speech › Information extraction and text analysis
named entity recognition
0.212022
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning · ACM Multimedia 2022

Methods — techniques the papers use, named apart from their topics

prompt tuning · 0.6fine-tuning · 0.6CLIP · 0.6BART · 0.6
YearPublicationVenuePosition
2022 Fine-tuning with Multi-modal Entity Prompts for News Image Captioning
abstract
News Image Captioning aims to generate descriptions for images embedded in news articles, including plentiful real-world concepts, especially about named entities. However, existing methods are limited in the entity-level template. Not only is it labor-intensive to craft the template, but it is error-prone due to local entity-aware, which solely constrains the prediction output at each language model decoding step with corrupted entity relationship. To overcome the problem, we investigate a concise and flexible paradigm to achieve global entity-aware by introducing a prompting mechanism with fine-tuning pre-trained models, named Fine-tuning with Multi-modal Entity Prompts for News Image Captioning (NewsMEP). Firstly, we incorporate two pre-trained models: (i) CLIP, translating the image with open-domain knowledge; (ii) BART, extended to encode article and image simultaneously. Moreover, leveraging the BART architecture, we can easily take the end-to-end fashion. Secondly, we prepend the target caption with two prompts to utilize entity-level lexical cohesion and inherent coherence in the pre-trained language model. Concretely, the visual prompts are obtained by mapping CLIP embeddings, and contextual vectors automatically construct the entity-oriented prompts. Thirdly, we provide an entity chain to control caption generation that focuses on entities of interest. Experiments results on two large-scale publicly available datasets, including detailed ablation studies, show that our NewsMEP not only outperforms state-of-the-art methods in general caption metrics but also achieves significant performance in precision and recall of various named entities.
Jingjing Zhang 0007, Shancheng Fang, Zhendong Mao 0001, Yongdong Zhang 0001
ACM Multimedia1